What OpenStack high availability covers and where it gets hard
When the API, database or message queue of a private cloud becomes a single point of failure, OpenStack high availability is a set of redundancy choices across the control plane, database, message queue and storage, plus a plan for host failures and maintenance. Each of the four layers needs its own design decision: how API endpoints are load balanced and how virtual IP addresses move between controllers, how the database cluster is built and replicated, how the message queue is clustered and which queue type it uses, and how the storage backend keeps data available when a node drops out. The work spans the deployment tooling that builds and repeats these choices, the network design that carries virtual IPs and cluster traffic, and the storage design underneath both.
The difficulty is that these layers do not fail independently. OpenStack services share state through the database and the message queue, so a control plane that answers on a second controller can still stall if the queue it depends on has lost its quorum. Quorum rules also differ by component. A clustered database typically needs a majority of its members to keep accepting writes, a quorum-based queue has its own leader election and replication behavior, and a Ceph monitor set needs a majority to keep the cluster map current. Three controllers do not give you three identical failure domains. Confirm the quorum behavior, minimum member count and failure tolerance for each component in the exact versions you plan to run, then test a controller loss in each layer before go-live.
High availability of the control plane does not protect tenant workloads. A failed compute host takes its running instances down with it unless the design includes a way to restart or evacuate them elsewhere, and that behavior is a separate decision from controller redundancy. Instances that run from local hypervisor disks are lost with the host, while instances booting from Ceph-backed volumes depend on the storage network and the Ceph cluster staying reachable. Ask the team to document, for each workload class, what should happen on a hypervisor failure: whether instances restart automatically, whether they are evacuated, whether tenants are expected to rebuild them, and which storage they depend on. Those answers drive host sizing, fencing and network design more than the number of API servers does.
Maintenance is where most HA designs are tested in practice. Draining a compute host, patching a controller or rebooting a database member should not interrupt tenants, but each action touches a different layer and a different set of services. Plan the maintenance sequence per layer, record which steps require tenant notice, and check that the cluster stays in quorum during each step. Upgrades add another constraint, because the release you run determines which upgrade paths are supported. The 2026.1 Gazpacho release is a SLURP release, and 2025.1 Epoxy and 2025.2 Flamingo remain in the maintained phase, so the upgrade plan and the HA design should be reviewed together so the maintenance sequence matches the upgrade path.
Effort and cost follow from the number of layers in scope, the number of host and workload classes that must be tested, and whether storage is shared with an existing Ceph cluster. Ask any partner for a written list of the failure scenarios they will test, the layers each scenario touches, the evidence they will hand over, and the items that depend on hardware or network decisions you have not made yet. Those open decisions, more than the controller count, usually determine how long the work takes and how much of it has to be repeated after the first design review.
Which OpenStack control plane and shared services need redundancy
A highly available OpenStack cloud is only as available as its weakest shared layer. The cards below cover the components that every service depends on, from the API endpoints in front of Keystone, Nova, Neutron, Glance, Cinder and Horizon to the database, message queue, cache and storage backend behind them.
API endpoints and load balancing
Keystone, Nova, Neutron, Glance, Cinder and Horizon each expose API endpoints that clients and other services call directly. If the only instance of an endpoint or its load balancer is lost, users cannot authenticate, launch instances or manage volumes, even when compute hosts are healthy. The usual approach is several API workers per service on multiple control nodes, with a virtual IP or load balancer distributing requests and health checks removing failed backends. Confirm the exact endpoint and health check settings against the deployment guide for your release.
Database holding service state
Every OpenStack service stores its state, such as instances, networks, volumes and identities, in a relational database. Losing the database node stops API calls that write state and can leave services unable to start. Common designs use a clustered database with replication and automatic failover, with a quorum mechanism to prevent split-brain writes. Ask your deployment tooling how it configures the cluster, how failover is tested, and how backups are taken and restored.
Message queue for service RPC
OpenStack services send remote procedure calls to each other through a message queue, so a failed broker breaks scheduling, networking and volume operations even when every API endpoint is up. Redundancy typically means a clustered broker with mirrored or quorum queues spread across control nodes. Check how queues behave during a broker node failure, whether in-flight messages are lost, and how clients reconnect after failover.
Caching layer
Services such as Keystone use a shared cache for tokens and other lookups. If the cache is lost, requests slow down or fail when a service depends on it for correctness. Decide whether the cache is clustered or shared across control nodes, how it is sized for peak authentication load, and what each service does when the cache is unavailable. Confirm the caching backend and its failover behavior against the deployment guide for your release.
Storage backend behind Glance, Cinder and Nova
Glance images, Cinder volumes and Nova guest disks depend on the storage backend. When Ceph block devices back these services, the Ceph monitors, managers and OSDs must keep quorum and replicas to serve I/O, so storage faults can stop instance boot and volume attachment across the cloud. Plan the replication factor, failure domains and network paths before deployment, and test how VMs behave when a storage node fails. Ceph integration design details appear in the Ceph storage topic.
Choosing the database, queue, virtual IP and storage approach
This table compares the four main high availability decisions for an OpenStack control plane and the compute storage behind it. Read each row as a trade-off: the option that fits a given team depends on the deployment tooling it has chosen, so confirm the tooling first.
| Decision | Option | When it fits | Operational cost |
|---|---|---|---|
| Database | Multi-controller database cluster with synchronous replication | Several controllers, a team comfortable running clustered database operations, and no tolerance for a single database failover step | Quorum management, split-brain handling, and careful maintenance ordering across controllers |
| Database | Single database with external failover | Smaller control plane, a team that already runs database failover tooling, and simpler day-to-day operations | A failover path to test regularly, and a short service interruption during failover |
| Message queue | Clustered message queue | Multiple controllers and services that depend on RPC and notifications staying up during a node loss | More nodes to size and monitor, and queue mirroring or quorum settings to verify per release |
| Message queue | Single queue node with restart and recovery procedure | Small environments where brief control plane disruption during a queue failure is acceptable | Less to run, but in-flight messages may be lost and recovery is manual |
| API endpoint | Virtual IP with a load balancer in front of controllers | Teams that want one stable endpoint address and a health-checked pool of API services | Load balancer pair to deploy, keepalived or equivalent configuration, and TLS handling to plan |
| API endpoint | DNS-based endpoints | Environments with a DNS team that can change records quickly and clients that handle name changes well | DNS change control in the failover path, plus TTL and caching effects to test |
| Instance storage | Shared Ceph storage with replication | Live migration and host evacuation are required, and the team can run Ceph alongside the compute fleet | Ceph cluster design, pool replication settings, and capacity planning that is separate from compute |
| Instance storage | Local hypervisor disks for instance state | Lower storage complexity and workloads that tolerate instance loss on host failure | Evacuation and recovery depend on rebuild from images, and data on a failed host must be backed up separately |
How to build OpenStack high availability in sequence
This sequence builds the control plane in layers, from network design through controllers, database, message queue, API endpoints and shared storage. Each layer is confirmed with a check before the next one starts, so the finished cloud can survive a controller restart or a single node failure without tenant downtime.
Confirm the network and virtual IP design
Your network team and the DeployOpen architect agree on the management, API, storage and tenant networks, the virtual IP addresses for the API and database endpoints, and which controller nodes sit on which switches or racks. The design is written down with addresses, VLANs and failure domains before any controller is installed. This step is done when every controller can reach every other controller and the virtual IP on the management network, confirmed with a ping and a TCP connection test from each node, and the client signs off the design document.
Deploy the controller set with your tool
The platform engineering team builds the inventory and global configuration for either Kolla-Ansible, which runs OpenStack services in Docker containers, or OpenStack-Ansible, which deploys into LXC containers or on bare metal. Use the 2026.1 deployment guides as the reference for the Gazpacho release, since 2026.1 is a SLURP release, and pin the same release in every inventory. Keep the inventory and configuration in version control so the deployment can be rerun. The step is done when the bootstrap and deploy runs finish without failed tasks and the Keystone and Nova APIs answer on each controller directly.
Configure the clustered database and confirm quorum
Deploy the database cluster across the controllers so the state for Keystone, Nova, Glance, Cinder and Neutron is stored on more than one node. Keep the cluster at an odd number of members so a majority can survive the loss of one node, and confirm with the deployment tool's documentation which clustering method it configures for your release. Check the cluster status from each member before moving on, because a node that joined with a wrong setting can look healthy while it is out of sync. The step is done when the cluster status query reports every member connected and in sync, and a test write made through one controller reads back through another.
Configure the message queue and confirm replication
Set up the message broker so RPC traffic between OpenStack services survives the loss of one node. Confirm the queue type and replication policy that your release and deployment tool use, then set them to match the availability target agreed in the design. Check that queues are replicated across members and that the broker's cluster view lists every node. The step is done when a queue declared through one node shows replicas on the others, and a service restarted during the test reconnects without manual intervention.
Front the API endpoints with a load balancer
Place a load balancer pair in front of the API endpoints and bind it to the virtual IP addresses from the first step. Terminate TLS where your security standard requires it, and configure health checks that remove a controller from rotation when its API stops answering. Test failover one controller at a time: stop the API services on one node, confirm that clients still receive responses through the virtual IP, then restore that node before testing the next. The step is done when a token request and a compute server list call succeed with each controller stopped in turn.
Connect Glance, Cinder and Nova to Ceph
Create Ceph block device pools for Glance images, Cinder volumes and Nova guest disks, and configure each service to use them as described in the Ceph documentation for OpenStack. Shared storage lets any compute host with access to the pools start an instance, so guest disks are not confined to one hypervisor's local storage. Confirm the Ceph cluster is healthy before connecting OpenStack to it. The step is done when a test instance boots from a Ceph-backed image, one controller is restarted, and the instance stays reachable with its attached volume still in use.
Planning host failures and maintenance without tenant downtime
Controller redundancy protects the OpenStack control plane, not the instances running on compute hosts. If a compute node fails or is drained for maintenance, the instances on it stop unless the platform moves them. Planning host failures therefore starts with a question for each workload: what happens to this instance when its hypervisor disappears, and who has agreed that outcome? Treat the answer as a platform decision that needs to be written down and communicated to tenants before the first incident.
Evacuation and shared storage. When a compute host is marked down, Nova can rebuild its instances on other hosts through evacuation, but the approach depends on where the guest disk lives. Instances booted from a Ceph-backed Nova disk or from attached Cinder volumes can be moved because every host sees the same storage. Instances whose disks sit on local hypervisor storage cannot be rebuilt elsewhere with their data intact. Inventory each flavor and image to find which guests fall into each group, and confirm with the official Nova documentation for your release how evacuation handles host failure, instance state and fencing before you promise recovery times to tenants.
Live migration. Live migration moves a running instance to another host without a full shutdown, which is the usual route for planned maintenance. Availability depends on the hypervisor, the CPU model compatibility across hosts, the storage backend, and any device passthrough, such as GPUs or SR-IOV ports, that the guest uses. Test live migration on each hypervisor and CPU generation in your fleet, and record which flavors pass. Do not assume a workload is migratable because the cloud supports the operation in general. Confirm the current behavior in the Nova and hypervisor documentation for your release and storage configuration.
Maintenance sequencing and headroom. Drain and patch one compute host at a time, and verify that the instances have landed and are healthy before moving to the next. Capacity headroom decides whether this works. A drain needs free vCPU, memory and storage on the remaining hosts, and the quantity needed depends on your own workload profile and anti-affinity rules. Model the worst case, where your largest host is out of service, and confirm that the remaining hosts can absorb its instances without breaching quotas or placement constraints. Revisit the calculation whenever you add flavors or change overcommit settings.
Workloads the tenant must re-architect. Some workloads cannot be protected by the platform. Instances with local-only state, pinned hardware, or single-instance designs with no tolerance for a reboot need the tenant to add redundancy across instances, zones or regions, or to accept a planned outage window. Agree on a classification with tenants, publish what the platform guarantees for each class, and record which applications need application-level failover. This list is the input that makes the maintenance and evacuation runbooks accurate.
What drives high availability effort and cost
This table lists the design choices and organizational factors that most change the build and run effort for OpenStack high availability, with how each one raises cost and how to keep it under control. Read it before asking for a quote, because the same scope can cost very different amounts depending on these inputs. One-time build work and recurring test and upgrade work are separated in the final rows so you can ask a partner to price each part.
| Driver | How it raises effort or cost | How to keep it down |
|---|---|---|
| Controllers and sites | Each extra controller adds quorum, load balancer and certificate work, and each extra site adds replication, witness and failover runbooks that must be built and tested. | Start with the controller count your failure model needs, and justify any second site with a stated recovery requirement. |
| Database and queue topology | Clustered MariaDB or Galera and RabbitMQ quorum queues need careful tuning, split-brain handling and monitoring, and each topology change means retesting failover. | Standardize on one database and one queue topology, and document the expected behavior for each failure type. |
| Shared storage and network throughput | Ceph or other shared storage needs replication sizing, dedicated network paths and recovery tests, and slow links extend rebuild and failover times. | Size storage replication and the storage network from measured workloads, and confirm rebuild times before committing to a design. |
| Services and integrations that must survive failure | Every service that must stay up during a host loss, such as Nova, Neutron, Cinder, Glance, Keystone and Horizon, adds configuration, testing and dependency mapping, and external integrations add their own failover paths. | Agree the list of services that need failover in the scope, and treat the rest as best-effort to limit test cases. |
| Number of environments | Lab, staging and production each need the same HA topology to make test results meaningful, so every extra environment repeats build and test effort. | Keep lab and staging smaller but topologically identical, and automate deployment so each environment is rebuilt from the same code. |
| Compliance requirements for failover evidence | Assessors may ask for documented failover tests, timings, logs and change records, which adds test scripting, evidence capture and repeat runs for each control. | Define the evidence list early, automate test output capture, and agree the test cadence with your compliance lead. DeployOpen prepares evidence but does not issue audit reports. |
| Team skills in clustered databases and queues | Teams without experience in Galera, quorum queues or Pacemaker-style fencing need more training, more partner hours and longer stabilization periods. | Assess current skills before scoping, and include knowledge transfer and runbook reviews in the written scope. |
| One-time build work | Topology design, controller and database build, queue and storage clustering, load balancer setup, initial failover testing and runbook writing are paid once at the start. | In the quote, list each build item separately so you can see what is included and what you would need to repeat later. |
| Recurring testing and upgrades | Every OpenStack upgrade, security patch or topology change requires regression testing of failover paths, and a new release may change HA behavior in one or more services. | Ask whether failover tests are automated and whether retesting per upgrade is included in a retainer or priced separately. |
How the HA layout changes by deployment model
This is the most common starting point. Several controller nodes run in one data center behind a virtual IP or load balancer, and they share one database cluster and one message queue cluster. The trade-off is simplicity: one network, one power feed and one change window, but a failure of the whole room takes the control plane down with it. Quorum is handled inside the site, so the central design question is how many members can be lost before the database or queue loses its majority. Keep the member count odd so a majority survives a single node loss. The deployment tool also changes placement. Kolla-Ansible runs services in Docker containers, while OpenStack-Ansible uses LXC containers or bare metal, so confirm the placement rules for your chosen tool before you set the host groups.
- Quorum: Keep database and messaging members on an odd count, each on a different host and power circuit.
- Shared entry point: Confirm the virtual IP or load balancer is highly available, since it is a common single point of failure.
- Test: Under load, power off one controller, then one database member, then one queue member, and record how long API calls fail.
- Maintenance: Run rolling restarts only after checking that the database and queue report healthy membership.
Before you commit, confirm that your tool supports the restart order and placement you plan to use.
Separate rooms or sites let the control plane survive the loss of one domain, but they add latency between members and create the risk of a split. When the link between domains fails, each side can see only its own members. With an even split, neither side holds a majority, and the cluster either stops or, if misconfigured, accepts conflicting writes. Quorum therefore has to be designed on purpose: decide which side keeps serving, where the tie-breaking member lives and what happens to writes when the link drops. Keep the overall member count odd, and do not assume that spreading members across two locations makes the cluster resilient on its own.
- Quorum placement: Decide in advance which domain keeps a majority if it loses contact with the other, and document it.
- Witness: Place a tie-breaking member in a third independent location so a link failure cannot produce two groups that each claim a majority.
- Latency: Measure round-trip time between domains against the database and queue requirements before choosing synchronous replication.
- Test: Cut the inter-domain link, then fail each domain completely, and verify which side keeps the API, database writes and message delivery.
Ask your deployment tool's maintainers and your storage vendor which of these placements they support for the release you plan to run, because the answer varies with storage design.
A small footprint uses fewer hosts, which lowers hardware, power and rack cost. The trade-off is contention: database, queue, API and network services compete with compute workloads for CPU, memory and disk, and a single host failure removes several roles at once. With few hosts, the odd-member rule may force database or queue members onto hosts that also run other control services. That is workable only if each quorum member still has a host to fail over to, and each member keeps enough reserved capacity to stay responsive under load. Co-location decisions should be recorded in the inventory so they are visible during every maintenance window.
- Resource reservation: Reserve CPU and memory for database and queue members so busy compute hosts cannot starve them.
- Quorum on shared hosts: Keep each database or queue member on its own host, even when API or network roles share hosts.
- Maintenance: A reboot of one host may take several roles down together, so plan workload evacuation and quorum impact as one task.
- Test: Reboot each host in turn during a realistic load test, then confirm the control plane recovers without manual database repair.
Confirm with your deployment tool which services it places on which hosts, since Kolla-Ansible and OpenStack-Ansible differ in how they place controller services, and check that placement against the quorum plan.
Securing the HA control plane: virtual IPs, TLS and replication credentials
Several concerns decide whether a highly available OpenStack control plane stays secure once it is running: how clients reach it, how its members authenticate to each other, who can change its routing, where its traffic travels and where its secrets live. The guidance below is specific to high availability. For the full hardening scope, including Keystone policies, hypervisor hardening and patching, see the OpenStack security hardening topic.
TLS on load-balanced API endpoints
Each API endpoint behind the virtual IP needs a certificate that the load balancer or service presents to clients, and controllers need trust settings for their internal calls. Certificates must include names that match the virtual IP and every controller hostname, or clients will fail validation after a failover. Schedule renewal so that all controllers receive the new certificate in the same window, and confirm with your deployment tool's documentation which services terminate TLS at the load balancer and which terminate it at the service.
Database replication and cluster credentials
Replicated database clusters and message queue peers authenticate members to each other with credentials that should be separate from service accounts. Create a dedicated replication user with the narrowest grants the cluster software documents, and limit which hosts may connect as that user. Record who holds these credentials and plan rotation so that every cluster member changes its password in one controlled window, which prevents a member that missed the change from rejoining with an old password.
Control of virtual IP and load balancer changes
A change to the virtual IP, its failover priority or the load balancer backend list can redirect every API request in the cloud to a different controller. Restrict write access to these settings to a small operator group, keep the configuration in version-controlled inventories, and require peer review before a deployment run applies it. Confirm which accounts can reach the load balancer management interface and whether those actions are logged.
Isolating management and replication networks
Management, API and database replication traffic should travel on networks that tenant instances cannot reach. Tenant traffic is handled by Neutron, so a misconfigured network, bridge mapping or security group can expose controller and database ports to guests. Map each HA traffic type to a named network, confirm that controller and database ports are closed on tenant-facing interfaces, and test reachability from a tenant instance after every network change.
Keeping queue and database secrets out of plain files
Database and message queue passwords often end up in deployment configuration files that many operators can read and that appear in backups. Store these values in an encrypted secrets store or the secret handling your deployment tool supports, and limit who can decrypt them. Check that rendered configuration files on controllers have restrictive file permissions and that no secret appears in logs or in the version-controlled inventory in plain text.
How to prove OpenStack high availability works before tenants depend on it
Do not tell tenants the control plane is highly available until a written test plan has shown each failure behaves as designed. Each test needs an expected result, the evidence to capture, and a stop rule that says what happens when the result differs. Run the plan first in a pre-production environment with the same topology as production, then repeat the critical tests in a planned window before onboarding tenants. Confirm the exact service names, health-check settings and tooling against your deployment method, whether Kolla-Ansible or OpenStack-Ansible, because the defaults differ between environments.
Start with controller failover, stopping one controller at a time. While one controller is down, run a scripted loop against the API endpoint behind the virtual IP or load balancer. Expect Keystone to keep issuing tokens, and expect Nova, Neutron, Glance and Cinder API calls to succeed, possibly with brief retries during failover. Also create, reboot and delete a test instance. Record endpoint probe logs, token request responses, load balancer and keepalived state, and the timestamps of each operation. If any API call fails for longer than your load balancer health-check window, stop the change, restore the node, and do not start the next controller test.
Next, test the database and message queue. For a clustered database such as MariaDB with Galera in many Kolla-Ansible designs, stop one node and confirm the remaining members hold quorum and keep accepting writes. Then test the case where quorum is lost, and confirm writes block rather than corrupt data. Record cluster size, quorum status and write results. For a message queue node failure, start operations that take time, such as instance boots, and record whether they finish or fail cleanly and can be retried. Stop the change if writes stall while a majority is still healthy, or if instances stay in transitional states after the queue recovers.
The storage network and compute host tests carry the most risk to tenant data. Interrupt the Ceph network path on one hypervisor for a short, agreed period. Expect running guests with Ceph-backed disks to pause I/O and resume without filesystem errors. Record guest I/O logs, Ceph health output and OSD states. Then power off a compute host. Confirm that Nova marks the service down, that fencing is in place, and that evacuation moves instances to healthy hosts without copying data. Record the evacuated instance list, new placements and any instance left in error. Do not evacuate without confirmed fencing, because a host that comes back while its instances run elsewhere risks split-brain disk writes.
Finish with a planned maintenance drain. Disable the compute service on one host, live-migrate its instances, confirm none remain, and then patch the host. Expect workloads to move with the migration results recorded for each instance class. Stop the change if migrations fail for a flavor or workload group, and do not drain the next host until the cause is known. Keep every result as dated, versioned evidence linked to its change ticket. These records support later compliance readiness work, but they do not by themselves satisfy any audit requirement, so confirm with your auditor what evidence they expect.
What to monitor, back up and document after go-live
A high availability design only holds if someone can see when it degrades, restore what it protects and repeat the failover tests after every change. The four areas below are where operating teams most often find gaps after go-live.
Monitor database and queue health
Track database cluster quorum state, member count and replication lag, and watch message queue cluster health and queue depth for the control plane services. A lost quorum or a partitioned queue can stall API calls while every individual service still reports as running. Set alert thresholds from your own service levels, not from defaults copied from another environment.
Monitor load balancers and virtual IPs
Check which controller holds each virtual IP, the health state of every backend in the load balancer pool, and whether failover moved the address as designed. Confirm that a failed backend is removed from rotation and that clients reconnect without manual steps. Record the expected owner of each virtual IP so a mismatch is visible at a glance.
Monitor storage health and replication
Watch Ceph cluster health, placement group states, OSD availability and recovery activity, since these determine whether volumes and images stay available during a controller or storage node failure. Track replication or recovery backlog after a node returns. Confirm alert routes reach the team that will act on them.
Track API latency and error rates
Measure response times and error rates for each OpenStack API endpoint behind the virtual IP, not only host-level CPU and memory. Latency that rises only during failover is an early sign of a misconfigured timeout or an overloaded backend. Compare results before and after each failover test to set a baseline your team can defend.
Back up databases and configuration
Schedule database backups and copy controller configuration and secrets to a separate location, then run a restore into an isolated environment on a regular cadence. An unrestored backup is an assumption, not a recovery plan. Record how long the restore took and which steps needed manual intervention, and feed that back into the runbook.
Back up Ceph volumes
Confirm how Cinder volume backups reach the Ceph backup pool and whether snapshots or incremental backups are used for each volume class. Test restoring a volume to a different host or pool, because that is the path you will need during a site or storage incident. Confirm the retention policy and who approves deletion with your storage team.
Rerun failover tests after upgrades
Repeat the controller, database, queue and storage failover tests after every OpenStack release change, patch level change and configuration change to the HA stack. Keep these tests as named steps in the upgrade plan covered by the upgrades topic, so they cannot be skipped under schedule pressure. Record pass or fail results against each step.
Document the controller inventory
Maintain a current inventory of controller nodes, their roles, virtual IP ownership, database and queue members and storage networks. Store it with the deployment configuration so that it is updated as part of every change. An accurate inventory is what allows an on-call engineer to answer which hosts must be restored first.
Write failover runbooks and version configuration
Write a runbook for each failure scenario, covering detection, decision owner, exact commands and the verification step that confirms recovery. Keep the runbooks and the deployment configuration in version control with review, so the document that operators follow matches the running system. Assign an owner to review each runbook after every test.
How much does OpenStack high availability cost and how long does it take?
What drives the cost of OpenStack high availability work?
The main cost drivers are how many parts of the stack must survive a failure and how many environments must be proven. Scope covers the controller nodes that run the API and scheduler services, the database cluster, the message queue, the storage backend (for example Ceph if it is in use), and compute host failure handling. Each component adds design work, configuration and failure tests. Environment count matters too, because a lab, a pre-production copy and production each need their own test runs and evidence. The size of your existing estate and your team's familiarity with the deployment tool also change the effort. Ask the partner to itemize the written scope by component and environment so that quotes can be compared on the same basis.
How long does an OpenStack high availability engagement usually take?
Duration depends on the same scope variables as cost, plus the calendar: hardware readiness, network design approvals and access to change windows for controlled failure tests. Failure testing usually runs in phases, and a failed test normally means a fix and a repeat, so the schedule is rarely linear. Ask for the timeline as milestones tied to deliverables, with the dependencies on your side named explicitly. A fixed-scope agreement should state the start conditions and what triggers a schedule change. Do not accept a duration that does not list its assumptions about environments, change windows and staff availability.
Can our in-house team build OpenStack high availability without a partner?
Yes, an experienced platform team can build OpenStack high availability in-house, and many do. The work requires working knowledge of the deployment tool you choose, the database and message queue clusters that OpenStack depends on, virtual IP and load balancer design, and a safe method for injecting failures. The real question is capacity and evidence: whether the team can build, test, document and then operate the design while handling its existing workload. Test a short list of these skills against your own staff before deciding. Also confirm whether your team can run failure tests in production-like environments without disrupting tenants.
What do we need to provide before OpenStack high availability work starts?
Plan for four inputs. First, hardware: enough hosts for controllers, database and queue nodes, storage and compute, sized, racked and reachable before the build begins. Second, network design decisions: management, API, tunnel and storage networks, VLAN layout, and how virtual IPs and bonding are handled, signed off by your network team. Third, access to change windows so that failure tests can be run and rolled back. Fourth, a named owner who accepts test evidence and can sign off on it. Confirm the OpenStack release early as well, since release status changes over time and the OpenStack releases site lists the current maintained series.
What is one-time and what recurs after OpenStack high availability is built?
One-time work includes design, build, configuration, failure testing, evidence collection and runbook documentation. Recurring costs keep HA working over time: repeating failure drills after upgrades, reviewing evidence when configuration changes, patching within maintained branches and staffing on-call support. Upgrades deserve particular attention. OpenStack stable branches are maintained for approximately 18 months, and only SLURP releases may enter an unmaintained phase afterward, so each upgrade can require retesting HA behavior. Ask the quote to separate one-time items from recurring ones and to state what triggers a retest.
What should a written, fixed-scope OpenStack high availability quote include?
A written, fixed-scope quote should list every component in scope with its environments, and the test cases for each failure mode, such as controller loss, database node loss, queue node loss, storage backend loss, compute host loss and a planned maintenance drain. It should name the deliverables, for example design documents, configuration kept in version control, test reports and runbooks. It should also list client responsibilities, milestones, acceptance criteria for test evidence, change-window arrangements and exclusions. Ask for assumptions in writing, how scope changes are priced, and a cost and schedule split by phase so the one-time and recurring parts are visible.
Which failure tests should we run before calling OpenStack highly available?
Test each failure you claim to tolerate, one at a time, under representative load, and record the result. Typical cases include losing one controller, failing over the database primary, losing a message queue node, losing a storage backend node, failing a compute host with instance recovery, and a planned maintenance drain. For each case, record what API clients saw, how long calls failed and whether the cluster returned to a healthy state. Check expected behavior against the documentation for your release and deployment tool, because results depend on configuration. Illustrative example: a controller failover that passes in a lab may still fail under production queue depth.
How DeployOpen scopes, builds and hands over OpenStack high availability
DeployOpen plans, implements, hardens, documents and supports self-hosted open-source platforms inside your own infrastructure, including OpenStack high availability for the control plane, database, message queue and storage. Engagements are fixed-scope, with scope and deliverables agreed in writing before work starts.
How we scope
- Map which control plane services, database, message queue and storage backends need redundancy in your environment.
- Record the failure cases the design must survive, such as a controller, a compute host or a storage node going down.
- Confirm the written scope, deliverables and timeline before any build work begins, and agree how maintenance windows will be handled.
How we build
- Deploy highly available API endpoints, database clustering and message queue layout with the deployment tooling your team chooses.
- Test failover for each component separately and document the observed behavior, including instance recovery after a host failure.
- Keep inventories, configuration and HA settings under version control so every change can be reviewed and reproduced.
How we hand over
- Deliver runbooks for controller failover, database recovery, message queue recovery and planned host maintenance without tenant downtime.
- Document the test results, known limits and the failure scenarios you should re-test after each upgrade or hardware change.
- Walk your operations team through the procedures and confirm they can run them without DeployOpen present.
How we support
- Ongoing support, upgrades and continuous testing are available as a retainer, scoped separately from the build.
- Re-run failover tests when OpenStack releases change, using the maintained branches that apply to your deployment.
- Confirm with us in writing which support responsibilities, response expectations and service levels apply before you rely on them.

