Hyperledger Fabric high availability starts with the ordering quorum
Availability in a Hyperledger Fabric network is decided mainly by the ordering service's consensus and by where its nodes are placed. Peer redundancy, Fabric CA availability and tested ledger backups matter, but they cannot compensate for an ordering layer that loses its quorum. A consortium or platform team that needs transactions to keep flowing through node, site or organization failures should settle the ordering design before anything else. The Fabric documentation recommends Raft, a crash fault tolerant ordering service that keeps working while a majority of its nodes are available. A five node Raft cluster therefore tolerates the loss of two nodes, and a three node cluster tolerates the loss of one. Fabric v3.0 added a Byzantine fault tolerant ordering service based on SmartBFT, which tolerates failures of up to, but not including, one third of ordering nodes. Choosing BFT means accepting more nodes, more operational complexity and a different failure model, so confirm the requirement with your consortium before choosing it. Solo and Kafka consensus were removed in v3.0, so any network still on Kafka must migrate to Raft before it upgrades. The harder problem is placement. Quorum arithmetic assumes that nodes fail independently, but in a consortium each orderer is usually run by a different organization on different infrastructure. The network is only as available as the least available organization that hosts enough orderers to break quorum. If two of five orderers sit with one member, that member's outage or policy decision can stop ordering for everyone. Ask each organization which orderers it will host, which sites and cloud accounts they sit in, and how many can be lost at once before quorum is gone. A healthy cluster does not protect against the failures that most often cause outages. An expired orderer or peer TLS certificate stops nodes from communicating even when every process is running. An unreachable Fabric CA blocks enrollment and renewal, and a restore that has never been rehearsed may fail when it is needed. Before go-live, verify that the quorum survives the loss of each planned node and site, that certificate expiry dates are tracked with owners, that the CA can be reached from every organization, and that a backup has been restored into a separate environment and the network has resumed ordering.
Decision to make: Document, per organization, the number of orderers it hosts and the failure it can cause before you finalize the topology.
Which Fabric components carry availability
A Fabric network stays available only if each of its moving parts keeps enough healthy members to do its job. The cards below cover the ordering cluster, peers, certificate authorities, health endpoints and the state that backups must capture, along with what breaks when each one is missing.
Raft or BFT ordering cluster
The ordering service orders transactions into blocks and decides whether the network can commit at all. Fabric documentation recommends Raft, which is crash fault tolerant and keeps working while a majority of nodes are available, so a five node cluster tolerates the loss of two nodes. Fabric v3.0 added a BFT ordering service based on SmartBFT, which tolerates failures of up to, but not including, one third of ordering nodes. Confirm the node count and consensus type with your consortium before go-live, because losing quorum halts ordering for every channel.
Peers per organization
Peers host the ledger, run chaincode during endorsement and gossip blocks to other peers. Each organization needs more than one peer so that endorsement and ledger reads continue when one peer fails, and so that gossip can still reach new or lagging peers. A single peer per organization turns a routine restart into an outage for that member. Check that endorsement policies can still be satisfied when any one peer is down, and that anchor peers are set on every channel.
Fabric CA servers and shared database
Fabric CA issues enrollment and TLS certificates, handles renewal and revocation and generates certificate revocation lists. Its default SQLite backend is single node, so a clustered setup needs PostgreSQL or MySQL shared by several CA servers behind a load balancer such as HAProxy. If the CA tier fails, new enrollments, renewals and revocations stop, and an expired certificate can take a node offline. Confirm the database itself is highly available, since a shared database that is a single instance moves the single point of failure rather than removing it.
Operations service health endpoints
Each peer and orderer exposes an operations service with a /healthz endpoint, a /version endpoint and a Prometheus /metrics endpoint or StatsD push, secured with mutual TLS client certificates. Load balancers and orchestrators use health checks to route traffic and restart nodes, and operators use them to spot a stalled node early. Without a working health check, a node that has stopped processing blocks can still appear up. Confirm which client certificates may query these endpoints and that probes can present them.
Ledger and MSP backup scope
Recovering a failed node or rolling back a bad upgrade depends on copies of the ledger and the MSP material for each organization, orderer and peer. The Fabric upgrade guide starts with backing up the ledger and MSPs before any binary changes. Backups should cover the peer and orderer file systems, the CA databases and the keys held in an HSM, where the HSM provider's own backup procedure applies. Test a restore on a separate host, because a backup that has never been restored is unverified.
Raft or BFT ordering, and how many nodes each network needs
This table compares the ordering, certificate authority, peer and release-line decisions that shape a Fabric high availability design. Read each row as one decision: the option, then the trade-off that drives node count, certificate work and operations. The main takeaway is that Raft is the documented default for crash tolerance, while BFT is a deliberate choice that requires v3.x and more operating effort.
| Decision | Option | Trade-off |
|---|---|---|
| Raft ordering | Documented recommendation. Crash fault tolerant with a majority quorum. A five node cluster tolerates the loss of two nodes. | Fewer orderers, certificates and alerts to run. Choose it when crash tolerance matches your consortium's threat model. |
| BFT ordering (SmartBFT) | Tolerates Byzantine ordering nodes, up to but below one third of nodes. Available only in v3.x. Confirm node counts in the current ordering documentation. | More nodes and certificates, closer consensus monitoring, and a v3.x-only feature your team must validate. Choose it when malicious orderers are in scope. |
| Single CA | One Fabric CA server, with SQLite as the default database. | Simplest to run, but an outage blocks enrollment and certificate renewal across the network. |
| Clustered CA | Several CA servers sharing a PostgreSQL or MySQL database behind a load balancer such as HAProxy. | Removes a single point of failure for enrollment. Adds a highly available database tier and load balancer to operate. |
| Peers per organization | Set by endorsement policy, gossip behavior and recovery targets for each organization. | Confirm the count with your architects and the Fabric documentation before sizing. Each peer adds hosting and monitoring. |
| Fabric 2.5 LTS | Long-term support release for production users. Patch releases continue in parallel with 3.x. | Conservative choice for production when BFT ordering is not needed. BFT ordering is not available on this line. |
| Fabric 3.x | Adds BFT ordering and Ed25519 MSP signing alongside ECDSA. Removes Solo, Kafka and the v1.x lifecycle. | Required for BFT ordering. Confirm the upgrade path and the current patch release before committing to this line. |
Building Hyperledger Fabric high availability step by step
This sequence takes a Fabric network from its current version and ordering setup to a state where ordering, endorsement, certificate issuance and recovery keep working when a node, site or organization fails. Each step ends with a check the team can record and repeat.
Confirm version line and migrate ordering
The network team records the Fabric version of every peer, orderer and CA, and decides whether the network stays on the 2.5 LTS line or moves to 3.x. Before any upgrade, Kafka ordering must be moved to Raft and every channel must use the v2.x chaincode lifecycle, because v3.0 removed Solo, Kafka and the legacy lifecycle. The check is a binary version report from each component, a channel listing that shows no legacy chaincode definitions, and a signed record of the chosen version line. Confirm the current patch releases with the Fabric release notes before scheduling changes.
Place orderers across sites and organizations
Plan the ordering nodes so that no single organization, data center or cloud account holds a quorum. Raft needs a majority of consenters to keep working, so a five-node cluster tolerates the loss of two nodes, and a placement that puts three nodes in one failure domain loses ordering when that domain fails. Agree the placement map with every consortium member, including who operates each node and which sites hold them. The check is a written placement table listing each orderer's organization, site and failure domain, with a review confirming that losing any one site leaves a majority.
Deploy Raft ordering with mutual TLS
Deploy the Raft ordering service on the agreed nodes, with mutual TLS enabled for cluster communication and client connections. If the consortium chooses the BFT ordering service that v3.0 added, confirm the node count against its tolerance for failures below one third of ordering nodes, which requires more nodes than Raft for the same failure count. Then verify that a leader is elected, that every consenter appears in the channel's consenter set, and that the orderers accept a test transaction. The check is the leader election and consenter membership recorded in each orderer's logs and channel configuration, plus a successful test block commit.
Add peers and confirm endorsement
Each organization deploys at least two peers so that endorsement and ledger reads continue when one peer is down, and the team configures anchor peers and gossip bootstrap addresses across the organizations. Each peer joins the channel and receives blocks through gossip. Confirm that every organization's peers appear in the gossip membership view of every other peer and that a test transaction is endorsed by peers from each organization named in the endorsement policy. The check is a membership listing from each peer and an endorsed, committed test transaction for each organization.
Cluster Fabric CA behind a load balancer
Run more than one Fabric CA server against a shared PostgreSQL or MySQL database, which replaces the default SQLite backend, and place the instances behind a load balancer such as HAProxy. Configure the CA servers to use the same database and identity registry so that any instance can issue, renew or revoke a certificate. Confirm enrollment by requesting a test identity through each CA instance directly and then through the load balancer, and check that the identity appears once in the shared database. The check is a successful enrollment result from every path, recorded with the test identity name.
Back up ledger and MSPs, rehearse restore
Set up scheduled backups of the peer and orderer ledgers, the MSP directories and the CA database, stored outside the systems they protect. A backup that has never been restored is not evidence of recovery, so run a restore into an isolated environment using the same Fabric version and configuration. After the restore, compare the restored ledger block height and the latest block hash with the source, and confirm that the restored peer rejoins the channel and resumes gossip. The check is a written restore log showing matching block height and hash, the time the restore took in your environment, and any gaps found.
Wire the operations service into monitoring
Connect each peer and orderer operations service to the monitoring platform, using the Prometheus endpoint or StatsD push and the mutual TLS client certificates the service requires. Confirm that the health endpoint reports healthy for every component, that the version endpoint matches the recorded release, and that metrics for ledger height and consensus state arrive in the monitoring system. Then stop one orderer in a test window and confirm that an alert fires for the loss of consensus participation. The check is a dashboard showing all components and a fired and resolved test alert with its timestamp.
What drives the effort of a high availability design
This table lists the drivers that move effort and cost in a Fabric high availability design, why each one matters and how to keep it down. Read each row as a decision to settle before the scope is written. Figures, durations and fees come from the written scope agreed with DeployOpen, not from this page.
| Driver | Why it moves effort | How to keep it down |
|---|---|---|
| Member organizations | Each member hosts its own nodes and approves consenter set changes, so every added member adds work. | Agree membership and the consenter plan before the build. Standardize one node profile that all members run. |
| Sites and independence | Nodes spread across sites with separate power, networks or cloud accounts need more infrastructure and runbooks. | Size failure domains to the fault tolerance the consortium agrees. Avoid sites that share one control plane or account. |
| Orderer and peer counts | Raft needs a majority of nodes. BFT needs failures below one third. Each node adds certificates, monitoring and upgrade steps. | Start with the smallest cluster that meets the agreed tolerance. Confirm counts against the ordering service documentation. |
| Fabric CA topology and database | SQLite is the default. Shared, highly available CA servers need PostgreSQL or MySQL and a load balancer such as HAProxy. | Reuse one CA database pattern across organizations. Confirm failover and backup behavior with the team that runs the database. |
| HSM for MSP keys | PKCS#11 keeps MSP keys in an HSM. Prebuilt images lack PKCS#11, so custom images must be built and maintained. | Decide early which keys go to the HSM. Use one image build pipeline for every node and keep TLS keys file-based. |
| Regulatory evidence | Assessors may ask for backups, access records, key handling and change history for every node. | Agree the evidence list at scoping. DeployOpen prepares evidence, while independent auditors assess it. |
| Consortium governance time | Consenter changes need member agreement, and aligning members can take longer than the technical work. | Agree a written change procedure up front, covering approvers, notice periods and emergency steps. |
| Backup and upgrade cadence | Fabric's upgrade steps begin with backing up the ledger and MSPs before rolling orderer and peer upgrades. | Script backups and rehearse rolling upgrades on a non-production network before each change. |
How to spread orderers across organizations and sites
Place orderers so that no single failure domain can take the ordering service below quorum. For a Raft cluster, that means counting failure domains rather than nodes. Two orderers in the same data center, the same cloud account or behind the same identity provider are, for most outages, one failure domain. If that domain fails, both nodes go down together, and a cluster that looked fault tolerant on paper stops ordering transactions. The planning question is how many orderers you need, and how many of them can fail together before the channel stops accepting blocks.
Governance matters as much as geography. Consenters are usually operated by different consortium members, and a Raft majority that one member can assemble on its own gives that member control over whether the channel keeps moving. Spread consenters so that no single organization holds enough of them to form a quorum alone. Agree on this allocation in the consortium charter before the first channel is created, because changing it later means changing channel membership under the consenter governance rules. Record which organization runs each orderer, who holds its keys, and who can authorize a replacement.
Sites and providers add a second layer of independence. Put orderers where power, network paths and identity systems are independent, such as separate facilities, separate network providers and separate administrative access paths. A site that shares a hypervisor cluster, a DNS zone or an SSO tenant with another site still shares a failure mode with it, even if the buildings are far apart. Ask each consenter operator to document its power supply, upstream network, DNS and certificate authority dependencies, and the people who can reach its management plane. Those answers tell you where the real domains are.
Latency and bandwidth between sites set the practical limit on how far apart consenters can be. Raft depends on the leader replicating entries to a majority of followers before it commits them, so high round-trip time between sites slows every transaction, and packet loss or unstable links can trigger repeated leader elections. Measure round-trip time and jitter between each pair of consenter sites under realistic load, and test a leader failover across the slowest link before you accept the topology. Confirm the supported Raft and BFT topologies and any latency guidance against the current ordering service documentation for your Fabric release, because the limits depend on the version you run.
A multi-site design adds network dependencies that must be monitored. Each inter-site link, its certificates, its DNS records and its firewall rules become conditions for consensus, so a silent break in one of them can look like a slow network instead of a failure. Set alerts on consenter health checks, on Raft leader changes and on replication lag, and link them to the operations service endpoints each orderer exposes. Plan the runbook for a lost site before go-live: who declares the site down, how quorum is restored, and how a returning node catches up. Test that runbook with the same consenter layout you will run in production.
Securing the high availability layer
Redundant orderers, peers and certificate authorities widen the attack surface as well as the availability margin. The cards below cover the security controls that change when a Fabric network is built for high availability, and they point to the hardening and certificate lifecycle topics for full detail.
Mutual TLS between nodes and clients
Orderers, peers and client applications should authenticate each other with TLS certificates, so a rogue process cannot join consensus or gossip. Each added orderer or peer adds another TLS identity to issue, rotate and revoke. Check that every node enforces client certificate verification and that the TLS CA trust chain is the one you intend.
File-based TLS keys with HSM-held MSP keys
Fabric can keep peer and orderer MSP signing keys in an HSM through PKCS#11, but TLS keys must remain file-based. High availability means every redundant node needs its TLS key available for restarts and failover, so the key storage plan must cover each replica. Confirm where TLS key files live and who can read them.
PKCS#11 limits and custom images
The prebuilt Fabric Docker images are not built with PKCS#11 support, so HSM-backed nodes need custom images. Each image rebuild must be repeated for every Fabric upgrade and patch release. Ask who owns that build pipeline and how HSM failover is tested before go-live.
Shared Fabric CA database and credentials
Several Fabric CA servers can share one PostgreSQL or MySQL database behind a load balancer, which makes that database the single store of identities and certificate state. Protect its credentials, restrict network access to the CA servers, and back it up with the same care as the ledger. Compromise here affects enrollment for every organization.
Restricting access to the operations service
The operations service exposes health, log level, version and metrics endpoints on each peer and orderer. Its mutual TLS client certificates should be issued only to monitoring systems and operators, and the endpoints should not be reachable from outside the management network. Changing log levels at runtime is useful during incidents, so limit who holds those certificates.
Encrypting and controlling backups
Ledger and MSP backups are needed before every upgrade and for recovery, and they contain material that can impersonate nodes or expose transaction data. Encrypt backup copies, restrict restore rights, and store keys separately from the backups. Test restores from the encrypted copies, not from the live environment.
Revoking lost node certificates
A lost or compromised orderer or peer must not be able to rejoin consensus with its old identity. Fabric CA handles revocation and publishes certificate revocation lists, but orderers and peers must be configured to pick up updated CRLs. Document who can trigger revocation, how quickly the channel configuration is updated, and how replacement nodes are enrolled.
Testing that the network fails over and recovers
A high availability design is only as strong as the tests that prove it, so write the test plan before go-live and run it against a staging network with the same ordering topology as production. Start with the orderers. Stop one orderer in a Raft cluster and confirm that the remaining nodes keep a leader and keep ordering. Use the operations service /healthz endpoint on each node and a steady stream of test transactions to confirm this. Record which node led before and after the stop. Then stop enough orderers to lose the majority and confirm that the channel halts. Submissions should fail or stall, and the surviving nodes must not produce competing chains. A halt is the correct result. Any divergence in block hashes is a failed test and a defect to fix before go-live.
Next, test recovery. Restart the stopped orderers and confirm that each one catches up to the current block height on every channel it serves. Compare heights across nodes directly instead of assuming that a running process has synchronized. Then stop one peer in one organization and send transactions that require endorsement. The transactions should keep succeeding through peers in other organizations, but only if the channel's endorsement policy can still be satisfied without that peer. Check the policy before the test. A policy that requires every organization turns a single peer outage into an application outage. Finally, restart the peer and confirm that gossip brings it up to the ledger height of its organization.
Test the certificate authorities as well. With clustered Fabric CA instances sharing one database behind a load balancer, stop one instance and attempt a new enrollment, a certificate reenrollment and a TLS certificate request through the balancer address. Each should succeed through the surviving instance. Confirm that revocation still works and that a revoked identity is refused on its next attempt. Also confirm that the database remains reachable from the surviving instance, since shared storage is a common single point of failure in this layout. Check the load balancer health probe during the test. A balancer that keeps routing traffic to a stopped instance will make the CA look down even when the cluster is healthy.
Restore testing is where many backup plans fail. Take a ledger and MSP backup from the test network, restore it into a separate environment with its own hosts, and bring up at least one orderer and one peer from the restored data. Verify that block height, the latest block hash and a sample of world state match the source at the time of the backup. Confirm that the restored MSP material still signs and validates for the restored identities. Treat a failed restore as a failed backup, even if the backup job reported success. Run the same restore before each upgrade. The Fabric upgrade guide begins by requiring a backup of the ledger and MSPs, and an upgrade that depends on an unrestorable backup leaves no fallback.
Record every test in a form that operators and auditors can reuse. For each case, capture the start and end times, the time to a new leader, the time until the first successful transaction after failover, the catch-up duration after restart, and the exact Fabric version and configuration under test. Record each outcome as pass, fail or deviation, with a reason. Keep the results with change records and runbooks. Repeat the full set after each upgrade, including rolling orderer and peer upgrades and channel capability updates, because a configuration that passed on the previous release can behave differently after a binary change. Ask your implementation partner for the test scripts and result templates as part of the written scope.
What to monitor, back up, upgrade and document after go-live
This table divides the recurring work of a highly available Fabric network between the consortium, which governs shared decisions, and the operator, which runs each node. Read each row as a task, then check who owns it and what evidence should exist. The main takeaway is that availability decays quickly without scheduled checks, tested restores and a documented upgrade path.
| Activity | What it involves | Who owns it |
|---|---|---|
| Operations service health checks | Poll each peer and orderer /healthz and /version endpoints, and scrape Prometheus /metrics (or receive StatsD pushes) over mutual TLS. Alert on lost Raft quorum, frequent leader changes and peer block heights falling behind the channel. | Operator runs the monitoring. Consortium agrees the alert thresholds and escalation contacts. |
| Runtime log levels | Use /logspec to raise log verbosity on a single node during an incident, then restore the default. Record each change so it does not stay on in production. | Operator |
| Ledger and MSP backups | Schedule backups of each peer and orderer ledger and of the MSP directories, stored outside the node's failure domain. Keep encryption keys for backups separate from the backup set. | Operator performs backups. Consortium sets the retention period and the recovery point each member accepts. |
| Restore testing | Restore a backup to an isolated host on a regular schedule and confirm the node rejoins and catches up. An untested backup is not evidence of recoverability. | Operator tests. Consortium reviews the results and signs off on recovery steps. |
| Upgrade path before each release | Follow the four steps in order: back up the ledger and MSPs, roll orderer binaries one node at a time, roll peer binaries one organization at a time, then update channel application capability levels. | Operator executes binary upgrades. Consortium approves the channel capability update because it changes network-wide rules. |
| Version line review | Track the 2.5 LTS patch releases and the 3.x patch releases in parallel. Note that 3.0 removed Solo, Kafka and the v1.x chaincode lifecycle, so confirm those are gone before any upgrade. | Operator monitors releases. Consortium decides which line the network stays on. |
| Consenter set and node placement records | Keep a current register of consenter certificates, orderer organizations, hosting sites and cloud regions for each node. Use it to confirm that quorum survives loss of one site or one organization. | Consortium owns the consenter set. Operator maintains node placement details. |
| Certificate expiry register | List enrollment and TLS certificate expiry dates for every node, admin and client identity, with the renewal owner and due date. Review it on a fixed schedule, since expired orderer or peer certificates cause outages. | Operator maintains the register and renewals. Consortium approves any change to identities in the channel configuration. |
| Runbooks and change records | Write runbooks for node restart, quorum recovery, certificate renewal, restore and rollback, with the commands, owners and evidence each step produces. Keep them with the configuration in version control. | Operator writes and tests runbooks. Consortium reviews those that affect shared channel or governance steps. |
Official Fabric references for this design
Before you settle the ordering topology, peer counts or CA layout, read the official Hyperledger Fabric documentation that governs each of them. The ordering service page explains how Raft stays available while a majority of nodes is up, and how the Byzantine fault tolerant option added in v3.0 sizes its cluster. Those rules set how many orderers each organization and site must host, so they belong in the design record before any infrastructure is ordered. Treat that page as the source for quorum arithmetic, and record the exact page version your team used so a later reviewer can trace each decision back to its source.
Upgrade behavior is governed by two pages: Upgrading your components, which describes rolling orderer and peer binary upgrades, and Upgrading to the latest release, which covers the migration steps Fabric v3.0 requires. The upgrade guide sequences a ledger and MSP backup before any binary changes, then updates application channel capability levels as a separate step. For a high availability design, check that each rolling restart still leaves a quorum of orderers online and that every organization keeps at least one peer available to endorse transactions. The What's new page lists the v3.0 changes you must absorb, including removed consensus types and the legacy lifecycle, so review it before choosing a target line.
Two operations pages affect how the design is secured and monitored. The hardware security module page explains that PKCS#11 can hold peer and orderer MSP keys, that TLS keys must stay file-based, and that the prebuilt Docker images do not include PKCS#11, so a custom image is required. The operations service page describes the /healthz, /logspec, /version and /metrics endpoints, which are protected with mutual TLS client certificates. Your load balancer and orchestrator probes will depend on those health endpoints, so confirm early how they will present client certificates. The Fabric CA user's guide covers the supported database backends and the shared-database pattern behind a load balancer, which is the starting point for clustering the CA.
The release pages decide which patch you deploy. The LF Decentralized Trust project page and the Fabric releases page on GitHub show the current lines. The Fabric documentation identifies v2.5 as the long-term support release for production users, and the 3.x line receives patch releases in parallel. At the time of research, the releases page listed v3.1.5 and v2.5.16, both published on June 18, 2026. Patch releases on either line can land at any time, so confirm the version you will deploy against those pages before finalizing the design, then read the release notes for that exact patch to check fixes that touch ordering, gossip or certificate handling.
Before sign-off, make three checks. Confirm that the documentation version matches the binary version in your target environment, because the default documentation tracks development. Record the release line and patch number in the design baseline so the high availability sizing, upgrade plan and HSM image build all refer to the same version. Finally, ask your implementation partner who will maintain the custom image and the patch cadence after go-live, since parallel patch streams mean the baseline will change over the life of the network.
Questions buyers ask about Fabric high availability
How much does a Hyperledger Fabric high availability design cost?
Cost depends on how many organizations take part, how many sites and ordering nodes the design spans, how many peers each organization runs, whether MSP keys sit in HSMs, and how much governance time the consortium needs before any build starts. We do not publish figures, and a quote follows from the written scope. Ask for the cost split into one-time work (design, build, testing, documentation, go-live) and recurring work (patching, certificate operations, monitoring, backup checks, HSM and infrastructure running costs). Also ask who pays for which component, since each member organization usually carries its own nodes, HSMs and staff. If a figure does not appear in the written scope, ask for it to be added before work starts.
How long does it take to design and build an HA Fabric network?
Duration is set mainly by the number of organizations that must agree on the design, the number of sites and environments, and how quickly the consortium approves the governance process and channel policies. Technical build time is usually shorter than the decision time. Other drivers include HSM procurement, custom image approval for PKCS#11 use, network access between sites, and the testing you require, such as failover drills and backup restores. The written scope should state phases, dependencies on member decisions and the milestones at which the consortium signs off. Ask the provider to name which dates depend on your side so the schedule is not built on assumptions.
Can our own team build a highly available Fabric network without outside help?
Yes, if your team already operates Fabric peers and orderers and the consortium has agreed how it will govern configuration changes, certificate issuance and upgrades. Without those two things, the technical work is rarely the bottleneck. Your team needs experience with Fabric CA, MSP and TLS configuration, the operations service, container and infrastructure tooling, and backup and restore. Many teams bring in a partner for the design review, a first build, or failover testing, then run the network themselves. Check your staffing against the roles in the written scope and confirm each member has named operators.
What does each member organization have to provide for the HA design?
Each member organization provides its own infrastructure for the peers and orderers it hosts, its own certificate authority and identities, and its own HSM if it uses one for MSP keys. It also provides named staff who approve design decisions, accept test results and run the nodes after go-live. Members must agree on shared decisions, including the Raft or BFT choice, orderer placement, channel policies and who may change them. Network access between sites, DNS, monitoring endpoints and backup storage must be available to every member. Ask for a written responsibility matrix that names the owner of each item before the design is signed off.
Should we choose Raft or BFT ordering for high availability?
Fabric documentation recommends the Raft ordering service for most networks. Raft is crash fault tolerant: it keeps ordering while a majority of nodes is available, so a five node cluster tolerates the loss of two nodes. Fabric v3.0 added a BFT ordering service based on SmartBFT, which tolerates Byzantine faults, meaning nodes that behave maliciously or inconsistently, up to but not including one third of ordering nodes. BFT needs more nodes, more coordination and more operational skill, so choose it only when your members do not trust each other's ordering nodes enough to accept crash-only failure. Confirm the exact node counts, supported versions and limits with the Fabric documentation and your own threat model before you decide.
What changes when we add a site or replace an ordering node?
Adding a site or replacing a node changes the consensus membership, so it is a controlled change, not a restart. You must update the channel configuration to add or remove the orderer, distribute the new node's certificates, confirm the cluster has a healthy quorum during the change, and verify that peers and clients connect to the new endpoint. Replacing a node also means retiring the old identity and its certificates. Plan a backup of the ledger and MSPs before the change and test it in a non-production environment first. Ask the provider to document the exact steps for your Fabric version, the quorum you must keep during the change, and the rollback path if the new node does not join.
How do Fabric CA and ledger backups fit into high availability?
High availability is not only about ordering. Fabric CA can run several servers that share a database behind a load balancer, so an enrollment or renewal service does not depend on one process. Choose the database backend with your operations team, since SQLite is the default and PostgreSQL and MySQL are the other supported options. Ledger and MSP backups are separate from consensus redundancy: replicas protect against node loss, while backups protect against corruption, bad changes and failed upgrades. Ask the provider to show how each backup is taken, where it is stored, how often it is tested by restore, and who signs off on restore procedures.
How DeployOpen scopes and delivers Fabric high availability
DeployOpen plans and implements Hyperledger Fabric high availability inside your own infrastructure, starting from the consensus choice and the placement of orderers, peers and Certificate Authorities across organizations and sites. Each engagement is fixed-scope, with the design, deliverables and timelines agreed in writing before work starts.
How we scope
- Confirm whether the ordering service will use Raft or BFT, and how many orderer nodes that choice requires for your fault tolerance target.
- Map which organizations host orderers, how many peers each member runs, and where each node sits across sites.
- Agree whether Fabric CA runs as a clustered service with a shared database behind a load balancer, and which backend it will use.
- Record backup scope for the ledger and MSPs, plus the recovery steps and who performs them.
How we build
- Deploy orderer and peer nodes across the agreed organizations and sites, with mutual TLS on every service-to-service connection.
- Configure peer redundancy per organization so endorsement and gossip continue when a single peer or site is unavailable.
- Test failover by stopping individual orderers, peers and CA servers, then confirming channel activity and certificate issuance continue.
- Verify operations service health checks, metrics and alerts so node failures surface quickly to the teams that run each member.
How we hand over
- Deliver a written architecture record covering node placement, consensus configuration, fault tolerance limits and failover procedures.
- Provide runbooks for orderer and peer replacement, CA database failover, ledger backup and restore, and certificate renewal.
- Walk each member organization's operators through failover tests and record the results in the handover pack.
- Document the backup schedule and restore test evidence so your auditors and consortium members can review them.
How we support
- Ongoing support, upgrades and continuous testing are available as a retainer after handover.
- Retainer scope can cover failover retests after Fabric patch releases on the 2.5 LTS and 3.x lines.
- Ask any quote to state which node failures, restore events and certificate renewals are included and which are billed separately.
- Clarify with the consortium who owns each node, its backups and its certificates, since these responsibilities drive ongoing effort.
