Choosing TechDocs storage: S3, Google Cloud Storage or Azure Blob Storage

Choose where production Backstage TechDocs sites live, weighing local storage limits, S3 and S3-compatible buckets, GCS, Azure Blob Storage, latency and separate CI and backend access.

On this page

Choosing TechDocs storage for Backstage in production

Generated TechDocs sites should live in external object storage, not on the Backstage server's local disk, once more than a handful of doc sites exist. The TechDocs architecture describes the basic setup, which serves docs from the Backstage server's local file system, and says that approach scales poorly as doc sites grow. The docs name AWS S3, Google Cloud Storage (GCS) and Azure Blob Storage as the external storage examples. TechDocs stores the built static HTML, CSS and assets for each entity, and the backend serves those files to readers, so the storage choice decides where the bytes live, how they reach the portal and who can write or read them.

Teams usually move off local storage when the portal runs in more than one replica, when doc sites are rebuilt in CI and need a shared destination, or when the server's disk is sized for the application rather than for documentation. Moving docs to a bucket also means the backend no longer needs write access to the filesystem for published content. The storage must be reachable from the backend, and the techdocs.publisher block in the app configuration selects the type. Confirm the exact publisher options for your Backstage version in the TechDocs documentation, and confirm with your cloud or storage team which provider already has the approved identity, encryption and logging controls.

The sibling pages cover how to publish from CI, how to limit visibility and how to cache responses, so this decision should focus on the provider and the location. Before choosing, record three facts: where the portal's backend runs, which provider your organization already operates and approves, and where your engineers and CI runners are. Those answers drive latency, egress cost and the separation of write and read credentials. If you run your own object storage, the S3-compatible options in the next section apply, but you must verify that the TechDocs publisher you need supports the endpoint and authentication method you choose.

Why local storage stops working as doc sites grow

The basic TechDocs setup builds and serves documentation from the Backstage server itself, which is fine for a few sites. The cards below explain where that model breaks down and what changes when sites are built in CI and published to a bucket.

The default serves from the backend disk

In the basic setup, the Backstage backend holds the generated HTML for every doc site on its local file system and serves it from there. Each site must be generated by the same server that serves it, so the server carries both the build work and the storage. Confirm in the current TechDocs architecture documentation whether your release still builds on the backend by default before you copy that setup into production.

Why the docs say it scales poorly

The TechDocs architecture documentation states that serving docs from the local file system scales poorly as doc sites grow. Every new site adds files to the backend disk, so capacity, backups and replacement of the server all grow with the documentation estate. Ask your platform team how much disk each backend replica can hold and whether replicas can share a copy of the same site.

Generation moves into CI

When sites are generated in CI, a pipeline runs the MkDocs build and publishes the output with techdocs-cli. The backend no longer needs to build anything, which keeps build tooling and credentials for source repositories out of the portal process. Plan the pipeline job for each docs repository, since it becomes the source of truth for when a site changes.

Publishing to a bucket

The TechDocs CI/CD guide documents the awsS3 and googleGcs publisher types for techdocs-cli publish. Once the output lives in a bucket, the backend reads finished sites from remote storage instead of its own disk. Confirm in the current guide which publisher types your release supports, including Azure Blob Storage.

Separate write and read access

Once CI publishes and the backend reads, two identities touch the same storage. CI needs write permission for its publish step, while the backend should only read. Decide how each identity is issued and rotated, and confirm the exact permissions each storage option needs before you grant them to either side.

S3, GCS or Azure Blob Storage: how the options differ

Amazon S3 has the most direct documented path in TechDocs. The TechDocs CI/CD guide lists the awsS3 publisher type for techdocs-cli publish, so a CI job can build a site and upload it to a bucket while the Backstage backend only reads. Plan the AWS account model before the first publish: which account owns the bucket, how CI authenticates to write, and how the backend authenticates to read.

  • Write path. The CI runner needs permission to put objects under the TechDocs prefix. Scope that role to the bucket or prefix and keep it separate from the backend role.
  • Read path. The backend needs read-only access to the same bucket. Grant only the actions it uses, so a compromised backend cannot overwrite published sites.
  • Account boundaries. Decide whether the bucket sits in the portal's account or a shared documentation account. That choice drives cross-account role design and audit scope.
  • Regional placement. Pick a region close to the backend and most readers. Measure latency from your own networks instead of assuming it.

Confirm the publisher options against the techdocs-cli version you run, and review bucket policy and encryption settings with your AWS security team before the first publish.

Storage options and when each fits

This table compares where generated TechDocs sites can live, how each option is published to, who maintains it, and what fits a production portal. Read each row for the fit column and the limits to check before committing. The main takeaway is that the Backstage publisher options cover AWS S3 and Google Cloud Storage directly, while Azure Blob Storage, Ceph RGW, MinIO and Garage need you to confirm the publishing path and feature support in your own environment.

Storage optionWhere data lives and publish pathLicensing and support modelWhen it fits
Local file systemFiles on the Backstage backend server; generated in the backend during the buildPart of Backstage, Apache-2.0; no separate storage productSmall doc estate or evaluation only; the TechDocs architecture says this scales poorly as sites grow
AWS S3Bucket in your AWS account; published with techdocs-cli using the awsS3 publisher typeManaged service from AWS; you pay AWS for storage and requestsTeams already on AWS that want managed durability and CI publishing with a documented publisher
Google Cloud StorageBucket in your Google Cloud project; published with the googleGcs publisher typeManaged service from Google Cloud; you pay Google for storage and requestsTeams on Google Cloud that want the same CI-to-bucket model with a documented publisher
Azure Blob StorageContainer in your Azure storage account; publisher path needs confirming for your Backstage versionManaged service from Microsoft Azure; you pay Azure for storage and requestsTeams on Azure; confirm the supported publisher and read path before choosing it
Ceph RGW (S3-compatible)Buckets on a Ceph cluster you operate, reached through the S3-compatible API; LGPL-2.1 or LGPL-3 sourceOpen source with no paid edition from the project; you own cluster operationsOrganizations that run Ceph on premises and need S3-compatible storage inside their own data centers
MinIO (archived community edition)Self-hosted S3-compatible store; repositories are archived and read-onlyAGPLv3 community source only, best effort support; commercial terms through AIStorAvoid for new builds; review AGPLv3 obligations if it already runs and plan a move to a maintained option
GarageSelf-hosted S3-compatible store across several sites; AGPL-3.0Open source; no bucket policies, versioning or object lock, so confirm your access and retention needsMulti-site or small on-premises setups that need only basic S3 features; verify the publish and read path with your Backstage build

Setting up external TechDocs storage

This sequence moves generated TechDocs sites out of the Backstage server's local file system and into a bucket your organization controls, with separate write and read identities and a measured check on page load from the backend region.

  1. Choose the bucket provider and region

    Decide whether the sites will live in AWS S3, Google Cloud Storage, Azure Blob Storage or an S3-compatible store you run yourself, and pick the region closest to the Backstage backend. Your platform team and cloud account owner make this decision, since it sets where documentation data is stored and which network path the backend uses to read it. Record the provider, region and account in the design notes. The step is done when someone can name the bucket location and explain why it was chosen over the alternatives.

  2. Create the bucket and two separate identities

    Create a dedicated bucket for TechDocs output, with no other data in it. Then create two identities: a write identity that only CI pipelines use to publish sites, and a read identity that only the Backstage backend uses to serve them. Grant write access only to the publishing path and read access only to the read path, and keep both identities out of developer laptops. Your cloud or storage administrator does this work. The step is done when a test write succeeds with the CI identity, a test read succeeds with the backend identity, and a write attempt with the backend identity is denied.

  3. Configure the publisher in app-config

    Set the TechDocs publisher in the Backstage app configuration to match the chosen provider, using the publisher type documented for that provider in the TechDocs CI/CD guide. The configuration names the bucket, region or account and the credentials source, and it should reference the read identity only. Keep secrets in environment variables or your secret store rather than in the committed configuration file. The step is done when the backend starts without publisher errors and its logs show the expected bucket name.

  4. Publish a test site with techdocs-cli

    From a CI job or a controlled workstation that holds only the write identity, run techdocs-cli to generate a small MkDocs test site and publish it to the bucket. Use a test entity name that no team depends on. Check the bucket contents afterward to confirm the expected structure appears under the entity's path. The step is done when the test site files are present in the bucket and the publish job exits without errors.

  5. Confirm the backend reads the test site

    Open the test entity's documentation tab in Backstage, signed in as a user who can see the entity. The page should render from the bucket rather than from the local file system. If it does not, check the publisher configuration, the read identity's permissions and the bucket path the backend requests. The step is done when the rendered page matches the test site and the backend logs show reads from the bucket, not local generation.

  6. Check page load time from the backend region

    Measure how long a TechDocs page takes to load in the browser, and compare the time to fetch objects from the bucket from a host in the same region as the Backstage backend. Note the results for a few representative pages, including a large one with many assets. The check tells you whether the storage location is adding noticeable latency and whether caching work will be needed later. The step is done when you have recorded load times and a clear comparison with the bucket region, and the team has agreed whether the placement is acceptable.

How the storage location affects page latency

The backend fetches every TechDocs page from the bucket, so the distance between the bucket and the Backstage backend shapes how long a page takes to load. When a developer opens a docs site, the backend requests the stored HTML and assets from the storage location and then returns them to the browser. A bucket in a region far from the backend adds a network round trip to each of those requests, and the delay grows with the number of files a page pulls in. Users far from the backend feel the same delay on top of their own network path. Placing the bucket in the same cloud region as the backend, and in the same region as most users where possible, is the simplest way to keep that cost low.

Do not assume figures from a vendor brochure or a region map. Latency depends on your network paths, any private connectivity, proxies and the provider's routing, so measure it from the place where your users and the backend actually run. Time a request from the backend host to the bucket with a simple tool such as curl against a known object, then time a full docs page load from a typical user location. Repeat the test at a busy hour, since the results often vary. Ask your cloud or storage provider which regions and endpoints serve your organization, and confirm whether the bucket endpoint your backend uses resolves to a nearby location.

Keep in mind that bucket placement is a separate question from caching. Moving the bucket closer to the backend lowers the cost of every fetch, but it does not stop the backend from repeating those fetches for every visit. Caching is covered in its own topic, so decide the bucket region first, record the measured baseline, and then decide whether caching is still needed. Put the bucket in the same region as the backend, document that choice in your deployment notes, and recheck it if you move the backend to another region.

Separating CI write access from backend read access

A TechDocs deployment has two parties that touch the same storage: the CI job that publishes generated sites and the Backstage backend that serves them to readers. Keeping their credentials separate limits what a leaked key can do and makes access reviews simpler. The cards below cover the principle, the failure modes and the provider-specific checks to make before rollout.

Write-only identity for CI

The CI job that runs techdocs-cli publish needs permission to create and overwrite objects under the TechDocs prefix, and nothing else. It should not be able to list every bucket in the account, change bucket settings or read other teams' artifacts. Scope the identity to the TechDocs bucket and prefix, and confirm with your CI platform how its secret is stored and who can read it. Keep the publish step on protected branches where possible.

Read-only identity for the backend

The Backstage backend only needs to fetch published files so it can serve them to users. Give it a separate identity with read permissions on the same prefix and no write or delete rights. If the backend is compromised, an attacker can read documentation but cannot replace it with altered pages, which matters because served HTML is what readers trust.

Risk of one shared credential

A single key used by both CI and the backend lets whoever obtains it publish arbitrary content that Backstage then serves under an internal address. Shared keys also tend to spread into config files, pipeline variables and developer laptops, so rotation becomes hard. Illustratively, a leaked CI token with write access could replace a runbook that engineers follow during an incident, which is why the two paths should not share a secret.

Bucket policies versus IAM roles

On AWS S3, the usual pattern is an IAM role for CI and a separate role or policy for the backend, combined with bucket policies that deny access from other principals. Google Cloud Storage uses IAM bindings on the bucket, often with separate service accounts. Azure Blob Storage uses role assignments on the container or account, with separate managed identities or service principals. The exact role names, condition keys and inheritance rules differ, so confirm each one against the provider's current documentation in your own account.

Workload identity over long-lived keys

Where the platform supports it, prefer short-lived credentials issued to the CI runner or the backend workload over static access keys. Check how your CI system federates to the cloud provider, how tokens expire and whether the backend runs with an identity attached to its compute. Ask your provider which of these options works with your hosting model before you commit to a design.

Verify and audit the split

After setup, test both identities directly: the CI identity should fail on read of unrelated buckets and on any delete call, and the backend identity should fail on every write. Enable the provider's access logging on the bucket and review which principal wrote each object. Record the results so an auditor can see that the separation was tested, not only configured.

Questions teams ask before choosing TechDocs storage

What does it cost to move TechDocs storage off the Backstage server?

The cost is mostly labor and cloud spend, and no vendor price applies to the Backstage side because Backstage is open source under the Apache License 2.0. The one-time work covers bucket creation, access policies, CI publishing and backend read configuration. Recurring costs are the object storage itself (stored bytes, requests and data transfer, billed by your cloud provider or by your self-hosted hardware) and the people who maintain CI jobs and credentials. Storage volume grows with the number of doc sites and their build history, so ask any partner to price the work from a written scope that names your bucket count, CI systems and cloud accounts. DeployOpen quotes follow from that written scope.

Can our team set up TechDocs storage without outside help?

Yes, if you already run CI, manage cloud identities and can read the TechDocs architecture and CI/CD guides. The core tasks are creating a bucket per environment, granting the CI job a write-only identity, granting the Backstage backend read-only access, and wiring the publisher type in techdocs-cli publish. The documented publisher types for techdocs-cli are awsS3 and googleGcs, so teams on Azure Blob Storage should confirm the current publisher options with the Backstage project before they commit to a design. Outside help is most useful when access separation, bucket policy review and multi-team rollout need to be designed together.

How long does TechDocs storage setup take?

Duration depends mainly on what already exists. A team with a working cloud account, an established CI platform and a single docs pipeline has the smallest job. Time grows when identity federation between CI and the cloud is missing, when several Git platforms or CI systems publish docs, when the bucket must sit in a specific region or data center, or when internal docs need visibility controls. No timeline is supplied for this work, so ask for a schedule that lists each of these dependencies as a separate milestone and confirm which ones your environment already meets.

What must our team provide before storage work starts?

Expect to provide cloud or data center accounts, the network path between Backstage and the storage endpoint, a list of Git repositories and CI systems that publish docs, and named owners for bucket policy approval. Specifically, you need bucket policies that give CI write access and the backend read-only access, the CI secrets or federated identities that let the publish step authenticate, and a decision on whether docs are public inside your network or restricted by catalog permissions. Keep write credentials out of the Backstage configuration and out of repository files. Agree this list in the written scope so gaps show up before build time.

Is self-hosted S3-compatible storage safe to run for TechDocs?

It can be, but the answer depends on the product and your operating capacity. Ceph's Object Gateway offers S3-compatible and Swift-compatible APIs and is licensed under LGPL-2.1 or LGPL-3, and its active releases receive periodic backports and security fixes. MinIO's community server is no longer maintained, its repositories are archived and its community binaries no longer receive updates, so it should not be chosen for new TechDocs storage without a support decision. Garage is an S3-compatible store licensed under AGPL-3.0 that lacks bucket policies and versioning, which affects how you lock down access. Whatever you pick, confirm the features TechDocs needs, budget for patching and backups, and treat the storage tier as production infrastructure.

Does storage location affect TechDocs page load speed?

Yes. Backstage serves TechDocs pages from whatever storage the backend reads, so each page view adds round trips to that location. Put the bucket in the same cloud region or data center as the Backstage backend where you can, and measure time to first byte from a realistic user location before you commit. Caching backend responses can reduce repeat reads, which is a separate design step covered on the caching page. Ask any partner to show measured latency from your own network, since no latency figure applies to your setup from this page alone.

How do we choose between S3, Google Cloud Storage and Azure Blob Storage?

Choose the object store in the cloud where your Backstage backend, CI runners and identity provider already run, because that keeps latency, identity federation and egress decisions in one place. Amazon S3 and Google Cloud Storage have documented publisher types in techdocs-cli, which simplifies the CI side. Azure Blob Storage is also named as an external storage example in the TechDocs architecture, so confirm its publishing path with your CI tooling before you choose it. If you run on-premises, compare S3-compatible options on support status and feature coverage rather than on cost alone. Write the decision down with the reasons, because storage moves are costly once many doc sites are published.

Should CI and the Backstage backend use the same storage credentials?

No. Give the CI publish job a write-capable identity that only publishes generated sites, and give the Backstage backend a read-only identity that only serves them. This separation limits damage if a CI job or the portal is compromised, and it matches the access split described in the TechDocs CI/CD guide. Use short-lived or federated credentials for CI where your platform supports them, store any static secrets in your secrets manager, and review bucket policies whenever a new doc site or publishing pipeline is added.

Security and operational considerations for TechDocs buckets

Once generated sites sit in a bucket, that bucket holds documentation that may be internal, and it becomes a second path to content that catalog rules protect. The cards cover the decisions to make before go-live: who can write and read, how data is encrypted and retained, how sites are restored, and who patches the storage software.

Separate write and read access

Give the CI pipeline a credential that can only write generated sites, and give the Backstage backend a different credential that can only read them. The TechDocs CI/CD guide describes publishing from CI, so the write path should live outside the portal server. Ask for separate credentials for each role, deny deletes and bucket listing to the reader, and set a rotation schedule you can name. Confirm which permission model your storage product enforces, because not every S3-compatible service supports bucket policies.

Encryption at rest and in transit

Confirm where encryption happens for each option. Ceph's messenger v2 secure mode encrypts traffic between cluster daemons with AES-GCM, which protects the storage backplane but not the path from the reader to the bucket endpoint. Garage supports SSE-C, where the client supplies the key, but it does not implement server-side encryption configuration. Ask each vendor whether keys are managed by you, by the platform or by an external key service, and who can recover data if a key is lost.

Retention and version history

Decide how long old builds stay and whether a bad publish can be rolled back. Garage does not implement bucket versioning or object lock, and its lifecycle rules support only AbortIncompleteMultipartUpload and Expiration. With that option, rollback has to come from rebuilding a previous commit in CI, and audit retention needs a separate archive. Ask each provider which of versioning, object lock and lifecycle expiration it enforces before you write the publish step.

Backup and restore of sites

A generated site is build output, so the source repository and the CI pipeline are the first recovery path. Rebuilding every site after a loss can take a long time on a large catalog, so ask whether the storage layer offers replication or snapshots. Garage stores each data chunk in three zones, and its deployment guide recommends at least three nodes, but replication does not protect against deleted objects. Test restoring one site from an empty bucket before go-live.

Patching self-hosted storage

Self-hosted bucket software needs a named patch owner. The MinIO community server's repositories are archived, its README says it is no longer maintained, and legacy binaries will not receive updates, which makes it a weak choice for a new production bucket. For Ceph, choose a release still inside its support window, because the Squid line's estimated end of life is October 31, 2026. Put the patch cadence, the test environment and the owner into the runbook before go-live. Check the Ceph active releases page for current dates.

Align bucket access with catalog visibility

TechDocs serves documentation from storage, so a credential that can read a site can also expose content that the catalog hides from some users. Decide whether one backend credential can read every site, and if it can, document that catalog permissions are the only gate. Review bucket policies and catalog visibility rules together whenever a team or entity changes ownership. Keep one source of truth for both so the two controls cannot drift apart. The sibling topic on restricting TechDocs visibility covers how TechDocs follows catalog permissions.

What a scoped TechDocs storage engagement covers

A scoped TechDocs storage engagement starts with an inventory of the current docs estate: how many repositories produce documentation, which catalog entities they belong to, which MkDocs plugins and extensions each build uses, how often sites change, and where the docs are served today. That inventory decides the storage design. A small set of sites built by one team on a single cloud account is a different problem from hundreds of sites with mixed ownership, and the written scope should say which situation applies before any bucket is created. Ask your DeployOpen contact to confirm the inventory method and the sample of repositories it will cover.

The written scope then names the chosen storage (an awsS3, googleGcs or Azure Blob Storage target, or an S3-compatible service you operate), the bucket or container layout, the region or site placement, and the encryption and retention settings. It also covers CI publishing with techdocs-cli and the pipeline credentials that write to the bucket, the read-only identity the Backstage backend uses to serve pages, and the separation between those two. The TechDocs CI/CD guide describes the publisher types that a scope like this will reference. Deliverables should list each role, policy and secret by name so your security team can review them against the bucket policy.

Handover is where most of the operational value is secured. The scope should state who receives the runbook, how a new docs site is onboarded, how a failed publish is diagnosed, and how storage costs and access logs are reviewed. Ask for the acceptance criteria in writing, including a test publish from CI and a page load from the portal. Engagements are fixed-scope: deliverables and timelines are agreed before work starts, and a quote follows from that written scope. Ask for each cost component to be itemized, separating one-time build work from any retainer for ongoing support.

How DeployOpen scopes and delivers TechDocs storage

DeployOpen plans and builds the storage layer behind TechDocs as part of a self-hosted Backstage deployment in infrastructure you control. The work covers where generated sites live, how CI writes them, how the backend reads them, and how access stays separated between the two.

How we scope

  • Written scope names the storage option chosen: S3, an S3-compatible bucket, GCS or Azure Blob Storage.
  • Scope records the cloud account, region or on-premises location and where the Backstage backend runs.
  • Scope states which publisher type CI uses and which read path the backend uses to fetch pages.
  • Engagements are fixed-scope, with deliverables and timelines agreed in writing before work starts.

How we build

  • Create buckets with versioning, encryption and public access settings reviewed against your policy.
  • Configure a write-only identity for CI and a separate read-only identity for the Backstage backend.
  • Wire the techdocs-cli publish step into your pipelines and confirm the generated site paths match the backend.
  • Test a docs build end to end, including a failed publish, a missing page and a permission denial.

How we hand over

  • Documentation records the storage decision, the bucket layout, the identities and their permissions.
  • Runbook covers rotating storage credentials, restoring a bucket from backup and changing regions.
  • Knowledge transfer sessions walk your platform and security teams through the configuration and tests.
  • Handover includes the test scripts so your team can rerun verification after upgrades.

How we support

  • Ongoing support, upgrades and continuous testing are available as retainers after handover.
  • Retainers can cover storage credential rotation, Backstage version bumps that affect TechDocs and provider changes.
  • Ask any partner to confirm response coverage and change handling in writing before signing.
  • Storage pricing and cloud or hardware costs stay with your provider and are outside DeployOpen's fees.

Talk to our team.

Tell us what you're working on, whether it's a deployment, an audit, a security test or a cyber range. You'll speak with an engineer who can help you scope it.

  • 30-minute call: free, with no obligation.
  • NDA on request: we can sign before you share details.
  • Clear next steps: a scope and plan after the call.