What a Prometheus rollout involves
Implementing Prometheus starts with an inventory of what you need to measure, and installing the server is the smallest step. List every host, service, database, queue and network device you check today, and note how each is checked. Each item becomes a scrape target that exposes metrics directly, a target reached through an exporter, or a check that stays with another tool. Teams usually start when ad hoc checks stop scaling, when a Kubernetes platform rollout needs metrics from day one, or when more teams need the same signals. Prometheus pulls metrics over HTTP, so the inventory also shows which endpoints the server must reach.
The design work sits in three places. Service discovery decides how targets appear and disappear without hand edits, which matters most where an orchestrator creates workloads. Exporter choices decide what gets measured and how many distinct series the server must hold. Storage is the third. The local time series database uses two-hour blocks and a write-ahead log, its default retention is 15 days, and it is a single node that is not clustered or replicated. Before you choose hardware, confirm how long metrics must stay queryable and how much disk that needs at your own sample volume.
The first working set is usually a small group of alert rules tied to symptoms that users or the business feel, plus Alertmanager routing that sends each alert to the team that owns the service. Grouping and silences keep notifications manageable, and Alertmanager can run as a cluster when notifications must survive a node failure. Dashboards come last and should be built in Grafana from metrics your teams already trust. Each of these pieces needs an owner who can review changes, which is where the people cost of a rollout usually sits.
Prometheus is Apache-2.0 licensed and has no paid edition, so the cost of a rollout is labor and infrastructure, and no license fee applies. A small team with a few targets and an on-call engineer can often build the first version in-house using the upstream documentation. Larger estates, multi-team alert ownership and audit evidence usually justify a partner, because those add review, testing and documentation work that grows with the number of teams. When you request a quote, ask for target inventory, exporter work, storage, alert design and handover to be priced separately.
What the work includes
A Prometheus rollout breaks into seven work packages, each with a defined output your team keeps. High availability, SSO, hardening, long-term storage and upgrades are separate topics with their own scope.
Target inventory and ownership
We list every host, service, endpoint and job that should be scraped, with the team that owns each one and the alert contact for it. The output is an inventory register that becomes the basis for every scrape job and alert route. Check that every target has a named owner before scrape work begins, because unowned targets tend to produce alerts nobody acts on.
Exporter and instrumentation selection
For each target we choose the exporter, client library or native metrics endpoint that exposes the signals you need, and record the version and the metric names it produces. The output is an exporter decision list with sizing notes on cardinality. Confirm with each application team which metrics they already expose, since custom instrumentation is often a separate scope.
Scrape configuration and service discovery
We set up scrape jobs using the discovery method that fits each environment: Kubernetes API, cloud provider APIs, file-based discovery or static targets. The output is a versioned configuration repository with relabeling rules and reload steps. Ask who will edit scrape configuration after go-live, since that person needs review access and a tested change process.
Retention and storage sizing
Prometheus keeps data in a local time series database on a single node, with retention set by time or size. We size disk and memory against your series count, sample interval and retention target, and document the result. The output is sizing notes with assumptions stated. Ask for the sizing model to be rerun with your measured series count, because estimates from design time often drift.
Alert rules and Alertmanager baseline
We write an initial set of PromQL alerting rules tied to the symptoms your owners care about, and configure Alertmanager grouping, silences and receivers. The output is a rule file in the configuration repository and a baseline routing tree. Plan to review the first set of alerts after several weeks of real traffic, since early thresholds are rarely right.
Dashboards and access
We build the dashboards agreed in scope and set the access model for the web UI and API, including which teams can query and which can change configuration. The output is a dashboard set under version control and a documented access list. Confirm how users will sign in, because Prometheus has no built-in single sign-on and that approach is a separate topic.
Handover runbooks
We write runbooks for restarting, reloading configuration, checking scrape health, handling a full disk and responding to the core alerts. The output is a runbook set and a walkthrough with the team that will operate Prometheus. Keep high availability, long-term storage, upgrades and hardening out of this package, since each is scoped on its own page.
What drives cost and duration
This table lists the factors that move Prometheus implementation effort and cost, why each one matters and how to keep it under control.
| Driver | Why it moves effort | How to keep it down |
|---|---|---|
| Number of targets and label cardinality | Every scrape target and every distinct label combination adds series to the local TSDB, which raises memory, disk and query load and the number of scrape jobs to write and test. | Inventory targets before scoping, drop high-cardinality labels at scrape or relabel time, and agree a series budget per team. |
| Mix of technologies needing exporters | Each platform (databases, network gear, hypervisors, message brokers, custom applications) may need a different exporter, configuration, authentication and dashboard set. | Prefer existing exporters where they fit, group similar systems into one scrape pattern, and defer niche sources to a later phase. |
| Service discovery platform | Integrating with Kubernetes, cloud provider APIs, Consul or file-based discovery requires RBAC, credentials and relabeling logic that must be tested in each environment. | Use one discovery mechanism where possible and document the relabeling rules that map discovered labels to standard job and instance names. |
| Required retention and query window | Local retention defaults to 15 days and can be changed by time or size. Longer windows usually require disk sizing, a remote write target and a query design for historical data. | Set retention to the window operations actually queries, and send only the longer-term data that audit or capacity teams need to a separate store. |
| Number of teams and alert receivers | Each team adds routing trees, receivers such as email, chat or paging tools, escalation rules and ownership questions that need sign-off. | Start with a small routing model, agree ownership for each receiver, and add teams in batches with a documented onboarding step. |
| Dashboard count | Dashboards must be built, reviewed against the metrics they use, and kept in version control as the metric names and labels change. | Limit the first release to the dashboards tied to agreed alerts and key services, and provision the rest from templates later. |
| Availability requirements | A single local Prometheus with one Alertmanager is simpler to build than redundant pairs, a clustered Alertmanager and a durable long-term store, each adding design, testing and operations work. | Confirm the outage tolerance for monitoring itself, then choose the smallest design that meets it and document the failure modes. |
| Compliance evidence needs | Assessors may ask for access logs, change records, retention settings, TLS and authentication configuration, and proof that admin APIs are restricted. Each item needs collection and a repeatable export. | Define the evidence list during scoping, use configuration as code, and confirm with your auditor which artifacts they will accept. |
| Number of environments and change control | Each environment needs its own configuration, secrets and test runs, and every rule or scrape change may need a review step, which lengthens each release. | Keep one reviewed configuration repository with environment overlays, automate rule validation, and agree a change window for alert changes. |
Project phases and outputs
The sequence moves from an agreed inventory of what Prometheus must monitor to a tested, handed-over system that your operating team runs without the implementation team.
Run discovery and target inventory
We list the hosts, services, endpoints and exporters Prometheus must reach, record how each one is discovered (static configuration or service discovery), and agree which alerts and dashboards matter to each team. The phase closes with a signed target inventory and a list of open questions for your network and security teams.
Settle architecture and sizing decisions
We decide the topology (single or paired Prometheus, Alertmanager layout, placement of exporters), retention period, storage sizing, authentication and TLS approach, and whether long-term storage through remote write is needed. The phase closes with a written architecture decision record that your architects and security leads approve.
Build in a non-production environment
We deploy Prometheus, Alertmanager, exporters, scrape configuration, recording and alerting rules, and dashboards in a non-production environment that you provide. The phase closes when every planned target scrapes successfully, rules load without errors, and a test alert reaches each receiver.
Run a parallel period and tune rules and cardinality
Prometheus runs alongside your current monitoring while we compare results, adjust scrape intervals, drop unneeded series, and correct noisy or missing alerts. The phase closes with measured series counts and storage growth reviewed against the sizing decision, and an agreed list of rule changes.
Cut over to production and review alerts
We move production targets and alert routing to Prometheus and Alertmanager under a change plan with a rollback step. Alerts are reviewed with the on-call teams during the first live period. The phase closes when agreed alerts fire and route correctly and the rollback path has been confirmed.
Hand over to the operating team
We deliver runbooks, configuration in version control, backup and retention procedures, the upgrade approach, and a walkthrough with the team that will operate Prometheus. The phase closes with signed handover acceptance and any remaining items recorded as the post-handover support scope.
What to keep in-house and what to outsource
This table compares each part of a Prometheus rollout with what teams usually keep in-house, what they usually hand to a partner, and the skills each part requires, so the split can be agreed before a quote is written.
| Work part | Usually kept in-house | Usually outsourced | Skills each part needs |
|---|---|---|---|
| Target inventory | Ownership list of services, hosts and owning teams; decisions on what must be monitored | Discovery of existing hosts, services and endpoints; drafting the inventory from configuration management and cloud accounts | Service ownership knowledge, cloud and CMDB access, familiarity with the applications being monitored |
| Exporter choice | Final call on which exporters are approved for each system and who maintains them | Evaluating exporters against each target, installing them and testing their metrics output | Operating system and middleware internals, Prometheus exporter conventions, change control practice |
| Scrape and discovery configuration | Approval of scrape intervals, labels and targets that will be queried by teams; access to secrets used in scrape config | Writing static and service discovery configuration, relabeling rules and TLS settings for scrapes | Prometheus configuration, service discovery for your platform (for example Kubernetes or cloud APIs), relabeling syntax |
| PromQL alert design | Defining which symptoms page a person and what each alert means for the service | Translating agreed alert definitions into PromQL rules and testing them against historical data | PromQL fluency, SRE practice for symptom-based alerting, understanding of service-level impact |
| Alertmanager routing | Routing tree, receiver list, escalation paths and on-call schedules that reflect the actual teams | Configuring grouping, inhibition, silences and receivers in the Alertmanager configuration and testing delivery | Alertmanager configuration, the notification tools in use, incident process knowledge |
| Storage sizing and retention | Retention period required by the business and any rules on where metric data may be kept | Estimating local disk and memory needs from series counts and sample rate, and setting retention and size limits | TSDB behavior, disk performance, capacity planning, and remote write design if long-term storage is added |
| Dashboards | Choice of what each team needs to see during an incident and sign-off on dashboard meaning | Building dashboards in Grafana or the chosen visualization tool, with panels and variables tailored to the services | Grafana or equivalent tooling, PromQL, interface design for operators |
| Security baseline | Policy decisions on TLS, authentication, who may reach the UI and API, and acceptance of residual risk | Enabling TLS and authentication on endpoints, keeping admin and lifecycle APIs disabled, and restricting exporters that accept URL targets | Prometheus security model, reverse proxy and certificate management, network segmentation |
| On-call runbooks | Written response steps for each alert, owner sign-off, and the people who answer pages and interpret what an alert means | Drafting runbook templates, linking alerts to runbook pages and reviewing wording with the on-call team | Incident response experience, familiarity with the services, clear technical writing |
What usually goes wrong
Most Prometheus rollouts that struggle do so for predictable reasons: unbounded label values, assumptions about storage durability, and operational gaps that appear after go-live. Each card names a failure mode and the check that prevents it.
Label cardinality growth
Each unique combination of label values creates a separate time series in memory and on disk. Labels such as user IDs, request paths or container hashes can multiply series counts quickly. Before go-live, review every metric's labels, set series budgets per job, and use relabeling to drop high-cardinality labels at scrape time.
Assuming local storage is durable
The storage documentation states that local storage is not clustered or replicated and should be managed like any other single-node database. A failed disk or node can lose recent data. Plan a second Prometheus instance, backups of the data directory, or remote write to a durable store, and confirm the recovery steps before relying on history.
Treating metrics as exact accounting
The Prometheus docs say the project values reliability over precision and is not suited to workloads that need 100% accuracy, such as per-request billing. Scrape gaps, rate calculations and sampling can all produce approximate values. Keep billing and audit records in systems designed for exact counts and use metrics for operational trends.
Exposed admin and lifecycle endpoints
Admin HTTP APIs are disabled by default, but enabling them or the lifecycle endpoints without access control lets anyone reload or shut down the server, or delete data. Confirm that these flags stay off in production, restrict the web listener to trusted networks, and require TLS and authentication on every endpoint that remains open.
Exporters that accept URL targets
Exporters such as SNMP and Blackbox take their targets from URL parameters. Anyone with HTTP access to these exporters can direct them to send requests to arbitrary endpoints. Restrict network access to the exporters, limit which Prometheus servers can call them, and review the scrape configuration for targets that users can influence.
Alerts without owners or runbooks
An alert that reaches a shared channel with no named owner tends to be ignored, and alerts without runbooks slow every response. Before go-live, assign each alerting rule to a team, attach a runbook link and severity, and route through Alertmanager receivers that match on-call rotations. Retire rules that nobody acts on.
Retention beyond the 15-day default
The default local retention is 15 days, configurable by time or size. Teams often assume months of history are available for capacity planning or incident review. Decide the retention each use case needs, size disk for that window, and choose remote write to external long-term storage when history must outlast the local database.
Skipping the parallel run
Running new alerts and dashboards without the existing monitoring for a period hides gaps in coverage. Targets missed in discovery, alerts that fire differently on PromQL, and thresholds that were never calibrated only appear under real load. Run both systems side by side, compare alert outcomes and cut over only after the differences are explained.
No plan for the 6-week release cycle
A new Prometheus minor release begins every 6 weeks, and a minor release generally stops receiving bug fixes after those 6 weeks. Without an upgrade routine, the deployment drifts out of support. Decide whether to follow the minor cycle or the LTS line, test rules and storage on each upgrade, and assign a named owner for the work.
How the work differs by starting point
Moving from a hosted monitoring service usually means the target inventory already exists, but the alerting logic and dashboards are tied to that vendor's query language and notification model. The work shifts toward translating those rules into PromQL and rebuilding how alerts reach on-call staff.
- Export or inventory existing alert rules, dashboards and notification integrations before estimating scope
- Confirm which hosted features (anomaly detection, managed retention, vendor agents) have no direct equivalent and need a design decision
- Plan local retention and the long-term storage path, since Prometheus does not replace a hosted service's history by default
- Ask whether the quote includes running both systems in parallel for a set period and the cost of that overlap
- Note that ongoing cost moves from a vendor subscription to compute, storage and staff time
Replacing Nagios or Zabbix checks changes the unit of work from host and service checks to scraped metrics. Each active check has to be mapped to an exporter, a blackbox probe or an instrumented endpoint, and each trigger has to become a rule that can be tested before it pages anyone. The migration page covers the detailed mapping.
- Count active checks, templates and triggers, since these set most of the rewrite effort
- Identify checks that depend on agent-side logic or SNMP traps, which need separate exporter decisions
- Plan a period where both systems alert, so thresholds can be compared before cutover
- Ask whether the quote includes rewriting alerts as PromQL rules or only deploying the server
- See the Prometheus migration topic for the check-to-exporter mapping and cutover steps
On Kubernetes, much of the target inventory is discovered from the cluster rather than listed by hand, so scope shifts toward labeling conventions, relabeling and the scrape configuration that depends on them. Effort concentrates in making discovery produce consistent job and team labels across namespaces.
- Agree on label and annotation standards for scrape targets before writing relabeling rules
- Decide whether to run the Prometheus Operator or plain configuration, since this changes how rules and scrape jobs are managed
- Confirm how many clusters are in scope, because each cluster may need its own Prometheus and federation or remote write design
- Include RBAC for service discovery and TLS for scrape endpoints in the security review
- Ask the vendor to show how new workloads are picked up without manual edits
In an air-gapped or restricted network, the work includes mirroring everything the deployment pulls from outside: container images, packages, exporters, rule libraries and documentation used during build. Updates follow the same path, so the upgrade process needs a defined transfer and verification step.
- Build an inventory of every artifact to mirror, including exporters and Alertmanager, and their update channels
- Plan an internal registry or package mirror and who approves transfers across the boundary
- Confirm that dashboards and notifications do not depend on external services such as public chat or SaaS endpoints
- Ask whether the quote covers offline testing of upgrades before production transfer
- Expect the cost to rise with the number of approval steps and the frequency of mirrored updates
A shared platform serving several teams adds scope around tenancy, alert ownership and access. The technical deployment may be similar in size, but the design work moves to who owns which rules, which receivers each team can change, and how dashboards are separated.
- Define alert ownership per team and the routing tree in Alertmanager that reflects it
- Decide how teams add scrape targets and rules without changing shared configuration, and how changes are reviewed
- Plan access control for the query UI and API, since Prometheus has no built-in single sign-on
- Estimate effort for onboarding each team, including training and sample rules, as a recurring activity
- Ask whether the quote includes a governance document and handover to team owners
What you need to provide
Before scoping starts, the client supplies the inputs that decide how large the Prometheus build is. The most important are access to hosts, Kubernetes clusters and cloud accounts for discovery, so the inventory reflects what is running rather than what the architecture diagram says. Pair that with a list of services and named owners. Each owner is the person who confirms which metrics matter, which alerts page them, and whether an exporter or application instrumentation is needed. Without owners, targets and alerts stay unassigned and the work stalls at handoff.
Network paths are the next input to confirm. Prometheus pulls from targets, so the server must reach every scrape endpoint, and every Alertmanager instance must be reachable from Prometheus. Ask for firewall change requests, segmentation rules and any proxies early, because they set the schedule more than the software does. Also state how Prometheus will be reached for queries and whether scrapes cross trust zones. Each path that needs approval adds review time and possible design changes.
Retention and query needs for each team drive storage sizing. Local storage keeps data in two-hour blocks with a default retention of 15 days, and that default can be changed by time or size. Ask each team how long it needs raw data, which dashboards and alerts depend on history, and whether anyone needs data beyond the local window. Long-term storage behind remote write is a separate cost and a separate design. Also collect the existing alert receivers and on-call tools, such as paging, chat and ticketing, so routing can be built and tested before cutover.
Three decisions must be settled before the build begins: the identity approach for access to the web UI and API, a named technical owner who can approve design and accept the handover, and change windows for cutover. Prometheus has no built-in single sign-on, so the team must choose an authenticating reverse proxy or accept basic authentication with TLS. The owner and change windows determine who signs the design and when production switches over, so confirm both with the client's change board before the quote is finalized.
Questions buyers ask
What does a Prometheus implementation cost if there is no license fee?
There is no license fee to compare because Prometheus is licensed under Apache-2.0 and has no paid enterprise edition, so the cost comes from infrastructure, storage, implementation and the people who run it.
One-time: the implementation itself, covering target inventory, exporters, service discovery, retention and storage sizing, alerting, dashboards and handover.
Recurring: compute, disk, any long-term storage, and the staff time needed to operate and upgrade the system.
Ask each written quote to separate one-time and recurring items, name who is paid for each, and state the assumptions behind the number of targets and storage sizing. DeployOpen does not publish prices, and a quote follows from the written scope.
How long does a Prometheus implementation take, and what sets the timeline?
The timeline depends mainly on how many systems and services need monitoring, how many exporters must be configured or written, how much alerting and dashboard work is in scope, and how quickly your teams review rules and notification routing.
Access approvals, network rules that allow scraping, and confirmation of service discovery sources often set the pace more than the Prometheus configuration does. Most engagements are fixed-scope, so the written scope sets deliverables and dates before work starts. Ask for a schedule tied to named milestones and to the inputs your team must supply, and check which of those inputs sit outside the project team's control.
What does the work need from our team, and can we do it alone?
Plan to provide an inventory of systems to monitor, access to those hosts or Kubernetes clusters, network rules that allow scraping, owners who approve alert thresholds, and the people who will receive notifications.
An internal team can implement Prometheus alone if it has working experience with PromQL, service discovery and on-call practice. Teams most often need help with exporter choices, relabeling, alert routing trees, high availability design, and securing endpoints with TLS and authentication, because Prometheus has no built-in single sign-on and SSO normally needs an authenticating reverse proxy.
Do we need long-term storage for Prometheus at go-live?
Not always. Local storage can cover go-live for many teams, because Prometheus keeps data for 15 days by default, and retention can be set by time or size.
Local storage is single-node, not clustered or replicated, so it should be run like any other single-node database, with backups and capacity monitoring. Remote write to an external long-term store becomes necessary when you must keep metrics beyond local retention or query across replicas. Set retention from your reporting and audit needs, then ask the quote to show how storage was sized.
Which Prometheus release line should we choose, minor or LTS?
Choose the LTS line if you want bug fixes for one year per release, or a minor release if you can upgrade on a faster cadence. A new minor release cycle begins every 6 weeks, and after those 6 weeks a minor release generally no longer receives bug fixes.
According to the Prometheus release cycle page, the current LTS line is 3.13, released July 1, 2026 and supported until July 31, 2027. LTS 3.5 reached end of life on July 31, 2026. Confirm the line against that page before go-live, and include the upgrade plan in the written scope.
What happens after handover of a Prometheus implementation?
Handover delivers the configured system, documentation, runbooks and the test evidence agreed in the scope. Your team can then operate it alone, or an ongoing support arrangement can cover upgrades, testing and continuous monitoring of the deployment.
Areas to confirm in writing include upgrade testing for alert rules and storage, Alertmanager changes, security configuration and who responds to incidents. DeployOpen offers ongoing support as a retainer, and its scope is agreed in writing. Ask the quote to list what is included at handover and what recurs afterward.
How DeployOpen delivers this work
DeployOpen plans, implements, documents and supports Prometheus in infrastructure you control, from target inventory and exporters through alerting and handover. Engagements are fixed-scope, with the scope, deliverables and timeline agreed in writing before work starts.
How we scope
- The written scope lists the targets, exporters, service discovery methods and alert sources to be onboarded.
- Retention, storage sizing and any remote write destination are defined in the scope before any build starts.
- Alert ownership, notification receivers and dashboard audiences are named so each deliverable has a clear owner.
How we build
- Configuration, service discovery and exporter setup are built and tested against a representative set of targets first.
- PromQL alerting and recording rules are written, validated against historical data and reviewed with the owning teams.
- Access controls, TLS and authentication settings are configured on each Prometheus and Alertmanager endpoint in your environment.
How we hand over
- Deliverables include configuration, rule files, dashboards and an operations guide written for your own engineers.
- Runbooks describe how to add targets, change alert routes, reload configuration and respond to a failing scrape.
- Handover includes a walkthrough with your operators so the team can run the deployment without DeployOpen present.
How we support
- Ongoing support, upgrades and continuous testing are available as a retainer after go-live.
- Upgrade testing covers recording and alerting rules and storage behavior before a new Prometheus release reaches production.
- Confirm with DeployOpen what a retainer covers, including response expectations, before signing the written scope.
