When a Graylog warm tier with searchable snapshots pays off
A warm tier keeps older Graylog indices on cheaper storage while leaving them searchable, so you can retain logs for months or years without paying hot-storage prices for every day of history. It earns its place when investigations or audits regularly reach back beyond your hot retention window, and when the volume of that older data is large enough that keeping it on the search backend's fast disks is hard to justify. If you only need a few weeks of searchable history, or you rarely search old data, a warm tier adds moving parts without much return, and a simpler retention policy or an archive may fit better.
Data tiering is an Enterprise capability according to the Graylog pricing page, so the first question is licensing, not storage. Graylog Open does not include tiering, and warm data is slower to search than hot data, which affects how analysts and automated event definitions should be designed. Confirm with Graylog whether your edition and Graylog Compute Unit or processed-data commitment cover tiering, and confirm the minimum number of hot days the feature requires in your version. Those answers determine whether the design is a warm tier, an archive with restore, or a hot-only setup with longer shard retention.
Before you size anything, settle three dependencies: the Enterprise license, the snapshot repository type (Amazon S3, Google Cloud Storage or a shared file system), and the search backend version. Graylog Data Node and self-managed OpenSearch have different compatibility rules, and the Graylog compatibility matrix is the place to check which backend versions your Graylog release supports. Ask your vendor or your own platform team to confirm repository access, retention controls such as object locking if your auditors expect them, and who owns restore operations. Once these are fixed, the storage math and tier settings become straightforward.
Warm tier repository options and when each fits
This table compares the repository types you can place a warm tier on, from cloud object storage to self-hosted S3-compatible stores and file systems. Read each row against your Graylog edition, backend version, and retention rules, since support for a repository type is only part of the decision. The main takeaway is that the storage choice also decides versioning, Object Lock, and who supports the software when something fails.
| Repository | Fit | Confirm before choosing | Main risk |
|---|---|---|---|
| Amazon S3 | Cloud-native, for teams already running in AWS | Your Graylog edition and backend version support S3 snapshot repositories, and whether versioning and Object Lock are needed for your retention rules | Costs and data residency depend on the region and storage class you pick, and Object Lock only works in versioned buckets |
| Google Cloud Storage | Cloud-native, for teams already running in Google Cloud | Repository support in your Graylog edition and backend version, plus the bucket versioning and retention features your auditors expect | Lock-in to one cloud provider, and the access model must be designed separately from the search cluster |
| Shared file system repository | Restricted network or air-gapped sites with an existing NAS or SAN | The path is mounted identically on every backend node and is listed in the backend's allowed repository locations | Single file system becomes a shared failure domain, and retention enforcement falls entirely to your storage controls |
| Ceph RADOS Gateway | On-premises object storage where Ceph already runs | The S3 API features you need (versioning, lifecycle, bucket policies) are available on your Ceph release, and Graylog accepts the endpoint | Object Lock is not listed among the documented features, so confirm it with Ceph before relying on it for immutable retention |
| SeaweedFS | On-premises S3 endpoint for many small files on your own hardware | Your chosen S3 features and Object Lock support, since Apache-2.0 Community use is separate from the paid Enterprise Edition | Support is community-based unless you buy the Enterprise Edition, so plan your own on-call and upgrade testing |
| Garage | Small geo-distributed clusters in restricted or edge sites | Whether your retention design can work without object versioning, object locking or server-side encryption | Garage does not support object versioning, object locking or server-side encryption, which weakens immutability evidence for retained logs |
| MinIO community (minio/minio) | Not recommended for new warm tier designs | Whether an existing deployment is already on a release you can keep patched from source | The repository is archived and no longer maintained, the last tagged release is from October 2025, and binaries are no longer published |
| MinIO AIStor (commercial successor) | On-premises object storage where you need vendor-backed support | Current licensing, support terms and the Graylog compatibility of the edition you would deploy | Licensing terms are commercial and separate from the AGPLv3 community server, so get the agreement reviewed before deployment |
Choosing a repository for cloud, on-premises and restricted networks
Cloud deployments usually place the warm tier snapshot repository in object storage from the same provider that hosts the Graylog nodes, which keeps the transfer path short and the operating model familiar. Amazon S3 Standard is designed for 99.999999999% durability and 99.99% availability, and it stores data across at least three Availability Zones, so a single zone failure should not remove the stored copies.
- Durability and availability: The storage classes documentation states the design targets for S3 Standard. Check that the class you choose for snapshot data fits how often your team will search restored or warm indices.
- Three-zone storage: Objects in S3 Standard span at least three Availability Zones within one region. That covers zone failures, but it does not cover a loss of the whole region.
- Immutability: S3 Object Lock offers governance and compliance retention modes and legal holds, and it works only in versioned buckets. Amazon states that Object Lock has been assessed by Cohasset Associates for SEC 17a-4, CFTC and FINRA environments, but your own compliance review still decides whether it meets your obligations.
- Account and region choices: Decide whether snapshots live in the cluster's account or in a separate, restricted account, and which region holds them. Confirm residency and cross-account access rules with your compliance owner.
Before you commit, check the repository type and version support for your Graylog release, the IAM role the Data Nodes will use, and whether the retention period conflicts with any Object Lock setting.
On-premises clusters often expose an S3-style endpoint through Ceph's RADOS Gateway or through SeaweedFS. Ceph RGW provides a RESTful API that is compatible with the basic S3 data access model, and Ceph documents support for versioning, lifecycle rules, multipart uploads, ACLs and bucket policies. The main constraint to plan for is replication: Ceph documents bucket replication only across zones, so a second site needs a multi-zone design, not simply a second cluster.
- Ceph RGW compatibility: The S3 compatibility covers the basic data access model, not every S3 feature. Test the exact operations Graylog performs against your RGW version before you rely on it.
- Replication limits: Plan a multi-zone layout with the storage team if snapshots must survive a site loss, because bucket replication does not work between independent clusters in the way some teams expect.
- SeaweedFS option: SeaweedFS is an Apache-2.0 distributed blob, file and S3 object store with a paid Enterprise Edition. Confirm which features your environment needs in the open source edition and which need the paid license.
- Operational ownership: The platform team owns patching, capacity, failure domains and backups of the object store itself. Snapshot data cannot be rebuilt from Graylog alone, so the storage layer needs its own recovery plan.
- Sizing: Snapshot volume follows the indices you move to the warm tier, so review storage sizing before you pick hardware or pool capacity.
Confirm with the storage vendor which S3 operations your version supports, and run a snapshot and restore test over the same network path Graylog will use.
Networks without outbound access usually cannot reach a cloud bucket or an external object store, so the repository is often a file system path on storage the Data Nodes can mount. That is straightforward to configure, but it places the recovery and immutability burden on your own storage processes. CIS Control 11, Data Recovery, asks for an isolated instance of recovery data, and it gives offline, cloud or off-site, version-controlled backups as example implementations.
- File system repositories: Confirm the mount, permissions and path requirements for your Graylog and Data Node versions in the official documentation before the build starts.
- Isolated copy: Keep at least one copy of snapshot data where a compromise of the production cluster cannot reach it, such as offline media or a segment with tightly restricted write access. This follows the intent of Safeguard 11.4 in Control 11.
- Versioning and retention: A plain file system has no built-in versioning or object lock, so retention and immutability must come from the storage platform, backup tooling or written procedures.
- Restore testing on the target network: Test restores where Graylog will run, including DNS, time synchronization, certificates and storage throughput, because a restore that works in a lab can fail in a restricted zone.
- Software sourcing: Plan how Graylog packages, Data Node or OpenSearch images and any storage software reach the network, and record the versions you mirror.
Before you rely on the warm tier for audits, check who can write to the repository, how the isolated copy is produced and how a full restore performs on the restricted network.
Setting up a warm tier in the order Graylog expects
This sequence takes a Graylog deployment from confirming that the license and search backend support tiering to proving that warm indices stay searchable, restorable and deletable on schedule. Each step closes with a check you can record before moving on. The order matters because a repository or version mismatch found late forces a rollback of indices that have already moved.
Confirm the edition includes data tiering
Check with your Graylog account contact, and against the current pricing and product pages, that your license includes data tiering. Graylog Enterprise adds data tiering alongside archive restore, so a Graylog Open deployment cannot use this workflow as documented. Record the license tier, the licensed volume and the support terms in the change ticket. Done when a license administrator has confirmed the entitlement in the Graylog license view and the record is saved with the project documents.
Match the search backend to the compatibility matrix
Identify whether the search backend is Graylog Data Node or self-managed OpenSearch, then compare the exact Graylog and backend versions against the Graylog compatibility matrix. Graylog 7.0 to 7.2 support OpenSearch from 1.1.x up to 3.7.x, and Graylog Security requires OpenSearch 2.0 or later. Graylog recommends Data Node, and OpenSearch 1.x use has been deprecated since Graylog 7.0 and is scheduled for removal in Graylog 8.0. Done when a written version pair is signed off, and any 1.x backend has a dated upgrade plan.
Register the snapshot repository on the backend
Create the snapshot repository on the search backend using the target type you have chosen, such as an S3 bucket, a GCS bucket or a file system path that every backend node can reach. Use a repository dedicated to Graylog warm data, and confirm that your backend version supports that repository type. Grant the backend only the bucket permissions it needs, and enable bucket versioning or object lock if your retention rules require it. Done when the repository passes the backend's verification from every data node and a test snapshot writes and lists correctly.
Set rotation and hot retention on the index set
In the index set for the log source, set the rotation strategy and retention so that indices stay hot for the period you choose before they become eligible to move. Rotation can be triggered by message count, size or time, and that setting decides how many messages each index holds before it closes. Confirm the minimum hot period in the documentation for your version, and set your hot period at or above that minimum. Done when the index set shows the intended values and a sample index closes with the expected message count and size.
Configure the warm tier policy
Attach the warm tier policy to the index set, setting the age at which indices move, the snapshot repository they use and any deletion that follows the warm stage. Apply the policy first to one non-production index set so the settings and their effects are clear before wider use. Record which indices are currently eligible and which remain hot. Done when the policy shows as active on the test index set and the exported configuration is saved in version control.
Run a test move on a non-production index set
Choose a non-production index set with several indices old enough to qualify, then let the tier move run or trigger it as your version's documentation describes. Watch the backend task status and cluster health during the move, and record how long it takes and how much load it places on the nodes. Stop and roll back if cluster health degrades or any index fails to transition. Done when each test index reports as warm, its snapshot exists in the repository and the cluster returns to its normal health state.
Verify search against warm indices
Run a search that spans hot and warm indices, and run the same query over a range that touches only warm data. If you captured result counts before the move, compare them with the post-move results and investigate any difference. Expect slower responses from warm data, and ask the log owners to confirm in writing that the slower response is acceptable for their investigations. Also check that dashboards and event definitions querying warm data complete as intended. Done when counts match the pre-move baseline and the owners have accepted the response times.
Restore and test retention in a scratch environment
Restore one warm index into a separate scratch environment with its own Graylog and backend, so the production cluster stays untouched. Compare message counts, a sample of message content and field mappings against the original index. Then let a test index pass its retention period on the test index set and confirm that the repository copy is removed according to the retention record. Done when the restored index matches the source and the deletion behaves as the documented retention policy states.
How minimum hot days and slower search trade off
The core decision is how long a log stays on hot storage before it moves to the cheaper warm tier. Hot indices serve the fastest searches, so every extra day kept hot adds to the storage bill, while moving data sooner lowers cost but makes older messages slower to query. The minimum hot period in Graylog's data tiering settings sets the floor for how early data can move, so it should be chosen from how investigations actually run, not from a default. Pull your analysts' search history for the last several quarters and find the window they query most often, such as the last 7, 14 or 30 days. That window is the one that should stay hot.
Warm queries are slower than hot queries because the data is no longer on local storage in the search cluster and must be served from the snapshot repository. That matters most when an investigation or an audit spans a long time range across many warm indices at once. A single broad search can then put heavy load on the search backend, slowing other users, dashboards and alerts that share the same cluster. Agree in advance who may run long-window searches, how far back they may go, and whether a restore to a searchable index is the better route for a large review. Test a representative long-window query on warm data before the tier goes into production.
The exact minimum hot period and its unit depend on the Graylog version and the tiering feature you use, so confirm both in the Graylog documentation for the release you will run, and check that the setting applies to your edition. Also confirm how the minimum interacts with index rotation and retention, since data cannot be moved before its hot period ends. Record the chosen value together with the investigation window it was based on, so future changes to retention are tested against the same reasoning.
Security and integrity controls for warm tier snapshots
A warm tier moves older Graylog data into snapshot repositories that sit outside the search cluster, so the repository becomes a second place where log data can be read, altered or deleted. The cards below cover the controls that keep that copy trustworthy, and each one ends with what your team should verify in its own environment.
Least-privilege repository access
Graylog needs credentials that can write and read snapshots in the repository, and those credentials should not be shared with anything else. Split them into one role that creates and deletes snapshots and a separate, narrower role that only reads them for restore. Verify that each role's policy allows only the bucket or path it needs, that keys are rotated on a schedule, and that no administrator account is reused for Graylog.
Encryption in transit and at rest
Snapshot data travels from the Data Node cluster to the repository and sits there as log content that may include personal data and credentials. Use TLS for every connection to the repository endpoint and enable server-side encryption on the storage itself. Verify that the endpoint certificate is trusted by every Data Node, that plain HTTP is disabled, and that your key management choice (provider-managed or customer-managed) matches your policy.
Object Lock modes and legal holds
For S3 repositories, Object Lock stores objects write-once-read-many, with governance and compliance retention modes plus legal holds. Governance mode lets users with special permissions remove protection, while compliance mode blocks deletion until the retention period ends, even for the account owner. Object Lock works only in versioned buckets, so enable versioning first. Verify which mode your retention policy requires and test a deletion attempt before relying on it.
Isolated recovery copies
A snapshot repository that is reachable with the same credentials as the live cluster can be damaged by the same attack that damages Graylog. The CIS Controls v8.1 Safeguard 11.4 asks for an isolated instance of recovery data, such as an offline, cloud or off-site copy with version control. Plan a separate account, bucket or site for recovery copies, and verify that a compromised Graylog host cannot delete or overwrite them.
Audit logging of snapshot and restore actions
Every snapshot creation, deletion and restore changes what evidence exists for a given period, so those actions need their own record. Turn on access logging or audit logging at the repository, and keep Graylog's own audit trail for who triggered snapshot and restore jobs. Verify that the logs are sent somewhere the Graylog administrators cannot edit and that you can match a snapshot name to the index and time range it covers.
Protecting MongoDB snapshot metadata
Graylog keeps the metadata in MongoDB that records which indices exist and which snapshots hold them. If that mapping is lost or altered, a snapshot can exist and still be unusable or pointed at the wrong data. Back up MongoDB on its own schedule, restrict access to the database to Graylog servers, and test a restore of both MongoDB and a snapshot together. Verify that the two backups are taken close enough in time to match each other.
What buyers ask about warm tier cost, effort and ownership
What actually drives the cost of a Graylog warm tier?
The main drivers are the volume of data you move to warm storage, how long it stays there, the storage backend you choose, and how much licensing and integration work sits around it. Storage is usually billed by the gigabyte per month for object stores, so volume and retention dominate the recurring cost. One-time costs include design, repository setup, index set changes, testing of restores and the time to document the runbook. Ask for the written scope to separate the one-time build from any retainer for ongoing support, and ask which storage costs you pay directly to your cloud or hardware vendor.
Do we need a Graylog Enterprise license for a warm tier?
Yes, the warm tier depends on Enterprise functionality. According to the Graylog pricing page, Graylog Enterprise starts at 15,000 US dollars per year, with a minimum of 10 GB per day or 100 Graylog Compute Units, and licensing is based on processed data with no per-user or per-device fees. Confirm the current edition terms with Graylog before budgeting, because the feature list and minimums can change. Graylog Open, which is licensed under SSPL-1.0, does not give you this tiering path, so your design should start from the license you hold.
How long does it take to build a warm tier on Graylog?
Duration depends on how many index sets you tier, how many storage repositories and access policies you must create, whether the Data Node or OpenSearch version is already current, and how much restore testing your auditors expect. A small pilot on one index set takes far less time than a rollout across all log sources with retention rules per team. Ask any provider for a timeline tied to a written scope with named phases, acceptance criteria for a test restore, and a list of assumptions about your environment. Be wary of any timeline quoted before someone has reviewed your ingest volumes and retention targets.
What does our internal team need to provide for the project?
Your team needs to supply the storage target and its access model, an account or service identity with rights to the repository, network paths between Graylog nodes and the storage endpoint, and a decision on retention periods by data class. You also need named owners for the Graylog cluster, the object store, and the security or compliance requirements that set retention and immutability. Expect to provide change windows for index set and Data Node changes and access to test data for restore checks. Confirm each of these items in the written scope so gaps are visible before work begins.
Can our own team run a warm tier without an outside partner?
Yes, a capable platform team can build and run a warm tier, provided it has Graylog administration experience, object storage operations skills, and time for testing. The work has several parts: sizing and index set configuration, repository setup, lifecycle and retention policies, restore procedures, and monitoring for storage errors and slow searches. Many teams find that the restore path and the upgrade path are where skills are thinnest. A partner can split the work into a fixed-scope build and a later retainer for upgrades, which you can compare against the staffing cost of keeping the skills in house.
How are restores from a warm tier handled, and how fast are they?
Restores bring older indices back from the warm tier, so searches against them are slower than against hot data and depend on the storage backend's latency and the load on your search cluster. Plan restores as a documented procedure with a named approver, a test restore before go-live, and a clear record of which data was restored and when. Ask the vendor or your own engineers to show a timed restore on a representative index set, and confirm how the restored data is removed afterward. Do not assume a restore time from a generic benchmark, because it depends on your data and hardware.
Who keeps the warm tier working after the first release?
Ongoing ownership covers Graylog and Data Node upgrades, storage backend patching, credential rotation, lifecycle rule reviews, and periodic restore tests. Graylog's compatibility matrix ties supported MongoDB and OpenSearch versions to each Graylog release, so each upgrade may require a backend change as well. Decide before go-live whether your team, a partner under a retainer, or a mix will handle those tasks, and record the responsibilities in a runbook. Ask for a written handover that names the alerts you will watch and the escalation path.
Limits to confirm in your environment before you commit
A warm tier only works if the edition, the Graylog and search backend versions, the storage product and your retention rules all line up. Each card below names one limit that can change the design, and the check that settles it before you commit budget or hardware.
Edition and repository support
Confirm that your Graylog edition includes data tiering and that the repository type you plan to use, such as S3, GCS or a shared file system, is supported for warm indices under that edition. Support depends on the license tier, not only on the Graylog version, so ask Graylog for this in writing against your exact license. Do not assume a feature in the product pages applies to every repository type.
Data Node and OpenSearch versions
Read the compatibility matrix for the Graylog version you plan to run and match it to your Data Node or OpenSearch release. Graylog recommends Data Node as the search backend, and use of OpenSearch 1.x is deprecated since Graylog 7.0 and is scheduled for removal in Graylog 8.0. If your current cluster runs OpenSearch 1.x, plan that upgrade before you add tiering, not after.
Minimum hot period and per-index rules
Find out how long an index must stay hot before it can move to the warm tier, and whether that rule applies per index set or globally. A short retention window may leave no room for a warm phase at all. Test with an index set that matches one of your real retention policies, and confirm how restored or rotated indices behave, because a rule that works for daily indices may not apply to larger rotation strategies.
Object store feature gaps
Check which capabilities your object store actually provides, such as versioning, Object Lock, server-side encryption and lifecycle rules. Some S3-compatible products omit one or more of these, and Garage, for example, documents no support for object versioning, object locking or server-side encryption. Where you need write-once retention for audit purposes, test Object Lock behavior on the exact product and version you will run.
Archived MinIO community repository
The minio/minio community repository is archived and no longer maintained, and it is distributed as source only with no new binaries. Legacy binaries will not receive updates, and the community server is under AGPL v3.0, which requires legal review for commercial or proprietary use. Before you choose MinIO, decide whether you accept an unmaintained build, a commercial edition, or a different S3-compatible product, and confirm the licensing position with your legal team.
Storage product licensing terms
Read the license of the object store and any client tool in the path, since terms differ across SeaweedFS (Apache-2.0 with a paid Enterprise Edition), Garage (AGPL-3.0) and rclone (MIT). Ask whether features your design relies on sit under the free license or a paid tier, and whether support is best effort or contracted. Record the answers in your design document so that a later upgrade or license change does not force a rebuild of the archive path.
How DeployOpen scopes and delivers a warm tier
DeployOpen plans, implements, documents and supports self-hosted open-source platforms inside your own infrastructure, so a warm tier is designed around your cluster, your object storage and your retention rules. The work starts with a written scope that settles the edition, the storage backend and the search backend before any configuration changes.
How we scope
- Confirm the Graylog edition and whether Enterprise licensing covers data tiering before design starts.
- Check the search backend version against the compatibility matrix, since Data Node or a supported OpenSearch release is required.
- Agree the repository choice (S3, GCS or file system), the minimum hot days and the search speed your investigators expect.
How we build
- Configure snapshot repositories, lifecycle rules and index set templates against a staging cluster with representative data.
- Test snapshot, move-to-warm and retrieval timings on your own indices, then record the results in the design document.
- Set object lock, versioning and access policies on the storage bucket to match the retention requirement you confirmed.
How we hand over
- Deliver a runbook covering tier transitions, repository health checks, failed snapshot handling and capacity thresholds.
- Walk your operations team through restore tests so they can run a search against a warm index without vendor help.
- Document the upgrade prerequisites tied to your Graylog and OpenSearch or Data Node versions for future planning.
How we support
- Engagements are fixed-scope, with deliverables and timelines agreed in writing before work begins.
- Ongoing support, upgrades and periodic restore testing are available as a retainer after handover.
- Confirm response expectations, change control and retainer terms in the written quote, since DeployOpen does not publish a price list.
