Argo CD Notifications for Slack, email and PagerDuty

Set up Argo CD Notifications so the right team hears about failed or degraded syncs in Slack, email or PagerDuty, with triggers, templates and secrets kept under your control.

On this page

How Argo CD Notifications alerts teams on failed or degraded syncs

Argo CD Notifications is the part of Argo CD that tells people when a sync fails or an application turns degraded, so the owning team hears about a broken deployment before a user does. A GitOps controller deploys quietly by design, which means a failed sync can sit unnoticed in the UI unless something routes it to a channel someone watches. Teams need this once more than one team or cluster depends on Argo CD, or once deployments happen outside business hours. Notifications is a feature of the open-source Argo CD project, not a separate paid product, so the cost is the engineering time to configure and test it.

The model has four parts. A trigger defines the condition, such as a sync that failed or a health status that became degraded. A template defines the message body for that condition. A service defines where the message goes, such as Slack, email or PagerDuty, along with the credentials it needs. A subscription connects an application or project to a trigger and a service, and it is set with annotations on the Application or AppProject resource. Triggers, templates and service settings live in the argocd-notifications-cm ConfigMap, and credentials for each service live in a Secret, so the routing rules can be reviewed in Git while secrets stay out of it.

Before you configure anything, confirm three things with your own environment: which services your on-call process actually uses, who owns each application so subscriptions map to a real team, and whether a degraded state on one application should page someone or only post to a channel. Test each trigger against a deliberately broken sample application in a non-production cluster, and confirm the message reaches the intended destination. The Argo CD notifications documentation covers the RBAC needed to manage these settings, which should be limited to the platform team so that alert routing cannot be changed silently.

Notification options and when each fits

This table compares the Argo CD Notifications building blocks, from triggers and templates to per-service configuration and subscription annotations on Applications and AppProjects. Read each row against your team count and cluster count to see where a simple setup stops being enough. The main takeaway is that subscriptions and destinations, not the trigger logic, usually decide how hard the setup is to govern.

OptionWhat it doesTrade-offWhen it fits
Custom triggers and templatesDefines your own conditions and message bodies in argocd-notifications-cmYou own the expression logic and must test each trigger against real sync statesTeams whose release process has its own failure or degraded criteria
Per-service configuration with SecretsStores Slack, email, PagerDuty or webhook credentials in a Kubernetes Secret referenced by the service entryEach new destination adds a credential to rotate and a service block to reviewSmall teams with one or two channels, or any setup that must keep tokens out of Git
Subscription annotations on ApplicationsRoutes notifications for one Application to a named service and channel or recipientPrecise, but every Application needs its own annotation, which drifts as the fleet growsSmall fleets where each owning team tracks a handful of Applications
Subscriptions on AppProjectsApplies a subscription to every Application in a project through a single definitionFewer edits and clearer ownership by team, but broad matches can send noise to the wrong channelMany teams and clusters that already separate work by project
PagerDuty for pagingRoutes failed or degraded syncs to an on-call service rather than a chat channelPages wake people, so trigger conditions and severity must be tuned before enablingProduction Applications with a defined on-call owner and a runbook
Email for audit and owner follow-upSends sync outcomes to distribution lists that owners or change reviewers readSlower to act on and easy to ignore if volume is high, so route only important eventsTeams that need a written record of sync outcomes for change review

Setting up Argo CD Notifications step by step

This sequence takes a running Argo CD installation from a version check to a tested alert on a failed sync, with each service credential, template and subscription confirmed before the next step builds on it.

  1. Confirm the Argo CD version and notifications controller

    Before changing anything, confirm which Argo CD version is installed and that the notifications controller is running in the argocd namespace. Run the version check with the argocd CLI or read the image tag on the argocd-server deployment, then check that the notifications controller pod is ready. Compare the version against the release line you plan to stay on, because the three most recent minor versions receive patch releases and the configuration keys can differ between releases. Confirm the target version with your release notes before you copy examples from any documentation. Done means you have a recorded version, a healthy notifications pod, and a named owner for the alert channels.

  2. Create service credentials as Kubernetes Secrets

    Create one Kubernetes Secret for each destination, such as a Slack bot token, an SMTP username and password, and a PagerDuty integration key. Keep these in the argocd namespace, restrict read access with RBAC, and never place tokens in argocd-notifications-cm or in Git. Reference the Secret keys from the notifications configuration using the $secret-name:key pattern so the controller resolves them at runtime. Verify by listing the Secrets and confirming each key exists with the expected name, without printing the values into a terminal log or ticket.

  3. Configure services and templates in argocd-notifications-cm

    Edit the argocd-notifications-cm ConfigMap to declare each service, such as slack, email and pagerduty, and reference the credentials created in the previous step. Then add templates that define the message body for a failed sync, a degraded application and a successful deployment. Keep templates short and include the application name, revision, sync status and a link back to the Argo CD UI. Check the syntax by applying the ConfigMap to a non-production cluster and reading the notifications controller logs for parse errors. Done means the controller starts without configuration errors for each service.

  4. Define triggers for failed and degraded syncs

    Add triggers to argocd-notifications-cm that fire on the conditions you care about. The usual choices are a sync that ends in a failed phase, and an application whose health becomes Degraded. Each trigger has a when condition written against application status fields and a list of templates to send. Keep the first set narrow so you can read every message that fires during testing. Verify the triggers by reviewing the rendered condition expressions with your team and confirming they match the health and sync values shown in the Argo CD UI for a deliberately broken test application.

  5. Add subscription annotations to a test application

    Pick one non-production Application and add a subscription annotation for each trigger and service pair you want it to use, following the notifications.argoproj.io annotation format in the official documentation. Include the destination name or channel for each service. Subscriptions set on the Application apply only to that Application, so this confirms routing before you scale it out. Verify by checking the annotations with kubectl or the UI and confirming the controller logs show the subscription being picked up without warnings about unknown triggers or services.

  6. Add subscriptions to a project for inherited routing

    Set subscription annotations on an AppProject so that every Application in that project inherits the same destinations, which keeps team routing in one place. Confirm with your team which project owns each channel, because a single project subscription can fan out to many applications. Check that the project annotation applies to the test application without duplicating the application-level subscription. Done means the test application receives its alerts from the project-level subscription, and you have recorded which channels each project feeds.

  7. Test a failed sync and confirm every channel

    Trigger a controlled failure in the test application, for example by pointing it at a manifest that references a missing resource in a disposable namespace. Watch the sync fail in the Argo CD UI, then confirm one message arrives in each configured channel with the expected application name, revision and status. Correct the manifest and confirm that the recovery produces the successful deployment template. Record the test result and the exact trigger and template names used. Only after this passes should you expand subscriptions to production applications, and keep the failure test as a repeatable check for future upgrades.

Writing triggers and templates that fire only on real problems

Triggers decide when a notification fires, and the message is shaped by a template that reads the application's state at that moment. In Argo CD Notifications, both live in the argocd-notifications-cm ConfigMap, and the template can reference fields from the Application object such as its name, namespace, project, sync status and health. Before you write anything, confirm the exact trigger and template names and the context variables available in your installed version against the project's notifications documentation, because these have changed between releases and a condition copied from an older example may not evaluate the way you expect. Test each trigger against a deliberately broken sample application in a non-production cluster and read the rendered message, not just whether something was sent.

The central trade-off is between alerting on every OutOfSync state and alerting only on failed syncs or degraded health. An application is OutOfSync whenever Git and the cluster differ, which happens during normal rollouts, manual hotfixes, and when automated sync is disabled on purpose. Alerting on that state produces a steady stream of messages that most recipients learn to ignore, and once they do, a genuine production failure is likely to be missed. A narrower default, such as alerting when a sync operation fails or when health moves to Degraded and stays there, is usually easier to defend. Decide with the application owners which states count as an incident, and reserve broader triggers for a separate channel that is reviewed rather than paged.

Noisy alerts erode trust faster than missing alerts are noticed, so review the signal regularly. Track how many notifications each channel receives per week, which applications produce repeats, and whether recipients act on them. Use the trigger conditions to suppress known transient states, and confirm with your team whether a retry or self-heal cycle should be given time before anyone is notified. Keep the template short and put the application name, target revision, the error message and a link to the Argo CD application page in every message, so the first responder can decide what to do without opening a second tool. Revisit the rules after each Argo CD minor upgrade, since trigger behavior and available fields are part of what your team needs to re-verify.

Securing service credentials and notification content

Notifications touch three kinds of sensitive material: service credentials that let Argo CD post to Slack, send email and page on-call staff, the configuration that decides who gets told what, and the sync details that end up in messages. The cards below cover where each one lives, who can change it, what goes into a message, and how changes are traced.

Keep service credentials in Secrets

Slack tokens, SMTP passwords and PagerDuty integration keys belong in a Kubernetes Secret referenced by the notifications controller, never in argocd-notifications-cm or a file committed to Git. Argo CD redacts cluster, Git and OAuth2 credentials from API responses and logs, but that protection does not extend to values you paste into a ConfigMap or a template. Check that no integration key appears in the Git repository that holds your Argo CD configuration, and confirm with Argo CD's security documentation which credential types it redacts.

Limit who edits notification config

Anyone who can edit `argocd-notifications-cm` can change where alerts go, including redirecting a PagerDuty route or adding a webhook that receives deployment details. Subscription annotations on applications and projects add a second edit surface that teams often control themselves. Restrict write access to the ConfigMap and Secrets to the platform group, and use Argo CD RBAC on Applications and Projects so teams can subscribe only to channels the platform has approved.

Keep sensitive sync output out of messages

Notification templates can include application names, revisions, commit messages and sync error text. A failed sync can print manifest fragments or environment values that were never meant for a broad channel or an email list. Review each template field before it reaches a shared Slack channel, keep detailed error output in the Argo CD UI for authorized users, and keep messages to status, application name, revision and a link back to Argo CD.

Encrypt outbound notification channels

Notifications leave the cluster for Slack, the mail relay and PagerDuty, so each outbound connection should use TLS. Confirm that the SMTP server you configure supports STARTTLS or implicit TLS, that the certificate chain is trusted by the notifications controller, and that the controller can reach the endpoint through your egress controls. In segmented or air-gapped networks, the internal relay and any proxy must be part of the design before the first alert is sent.

Scope tokens to the minimum needed

Each integration should use its own credential with the narrowest permission that still allows the notification. Create a dedicated Slack app or bot token limited to the channels it posts to, a separate SMTP account that can send only from the sender address you choose, and a PagerDuty integration key per service rather than one shared key. When an integration is retired or a person leaves, you can revoke one credential without breaking the others.

Audit and review notification changes

Notification behavior changes when someone edits a template, a trigger, a subscription annotation or a Secret, so these changes need the same review as application code. Keep the ConfigMap and subscription annotations in Git with pull request review, and enable Kubernetes audit logging for changes to Secrets in the Argo CD namespace. Ask your team which events in your environment are logged, who receives the logs, and how long they are kept.

Questions teams ask before setting up Argo CD notifications

What does setting up Argo CD notifications cost, and what drives the price?

Cost comes from the engineering time needed to design triggers, write templates, create service credentials and test alert routing, plus any partner fee for that work. Most of the effort scales with the number of applications that need subscriptions, the number of channels (Slack, email, PagerDuty), and how many escalation rules you need. Existing PagerDuty routing and on-call schedules also add work if they must be mapped to Argo CD. Ask any partner for a fixed-scope quote that lists each channel, each trigger type, the number of applications covered, and whether ongoing tuning is included. Notification setup is mostly one-time work, while credential rotation and template changes recur.

How long does Argo CD notifications setup usually take?

The duration depends on scope rather than on Argo CD itself. A single channel with a few triggers on a small number of applications is much shorter than a multi-channel setup with escalation paths across many teams. The main timeline drivers are how quickly your Slack app, SMTP relay and PagerDuty service can be approved and created, how many teams must agree on alert rules, and how much testing you need against real failed and degraded syncs. Ask for a written timeline that separates the access requests from the configuration and testing phases.

What must our team provide before notifications can be configured?

Before work starts, you typically need to provide three things: a Slack app with permission to post to the target channels, an SMTP relay your cluster can reach and authenticate against, and a PagerDuty service with an integration key for the teams that receive pages. You also need access to the cluster where Argo CD runs, permission to edit its ConfigMaps and Secrets, and a list of which applications and projects should send alerts to which destinations. Confirm with each vendor how their app or integration credentials are issued and rotated, because that decides who holds the secrets.

Can one in-house engineer set up Argo CD notifications alone?

Yes, a platform engineer who already operates Argo CD can usually configure a single channel alone, because the core work is editing the argocd-notifications-cm ConfigMap, storing credentials in a Secret and adding subscription annotations to Applications or AppProjects. The harder part is agreeing on alert rules, avoiding noisy triggers and getting the on-call and channel owners to sign off. A single person can build it, but a second reviewer is useful for testing failure paths and for checking that secrets are not exposed in logs or manifests.

How do we keep alerts working after an Argo CD upgrade?

Upgrades can change how Argo CD loads configuration, so alerts should be part of your upgrade test plan rather than an afterthought. Read the official upgrade notes for the target version, then verify that triggers still fire, templates still render, and each Slack, email and PagerDuty destination still receives a test message in a non-production cluster. Keep the notification ConfigMap and Secret definitions in Git so changes are reviewed and reproducible. Ask your partner whether post-upgrade alert checks are included in a retainer or handled as a separate task.

How do we know when a sync has failed or degraded?

Argo CD Notifications works through triggers that fire on application state changes, such as a sync failure or a health status that is not Healthy, and templates that format the message for each channel. You define which triggers matter, then subscribe Applications or projects to them with annotations. Test each trigger by forcing a failed sync in a non-production environment and confirm that the right channel receives it. Also decide who acts on degraded versus failed states, because a trigger that pages for every transient state will train teams to ignore alerts.

Where should notification secrets live so they stay protected?

Store Slack tokens, SMTP credentials and PagerDuty integration keys in Kubernetes Secrets in the Argo CD namespace, and reference them from the notifications configuration rather than placing values in the ConfigMap. Restrict who can read those Secrets with RBAC, and keep the plaintext values out of Git. Confirm with your security team how the secrets will be created (manually, through a secrets operator or through a sealed approach) and how rotation will be handled. The Argo CD security documentation notes that it redacts credentials from API responses and logs, but that does not replace controlling access to the Secrets themselves.

Official Argo CD sources to read next

The security documentation is the first read for anyone who will store Slack webhooks, SMTP credentials or PagerDuty keys in the Argo CD namespace. It explains how Argo CD protects the credentials it holds and where its own components expose risk, which matters because a notification template can read application status and link back to cluster details.

  • Credential redaction: Check what Argo CD redacts from API responses and logs, and confirm that your alert payloads do not add sensitive fields of their own.
  • Token-only API access: Confirm how JSON Web Tokens are issued and scoped before you grant any automation account access to notification settings.
  • Repo-server exposure: Review the warning that the repo-server runs Helm and Kustomize on repository content, since template changes in Git can change alert behavior.
  • Component TLS: Confirm which connections are encrypted between Argo CD components and what your outbound notification endpoints require.

Before you go live, check that every secret referenced by the notification configuration is limited to the namespace that needs it and that your security team has reviewed the list of outbound destinations.

How DeployOpen scopes and supports Argo CD notifications

DeployOpen plans, implements, documents and supports Argo CD Notifications inside your own infrastructure, so alert routing stays under your control. Each engagement starts with a written scope that names the channels, triggers, applications and projects to be covered before any configuration changes are made.

How we scope

  • Inventory which Slack channels, email lists and PagerDuty services should receive sync and health alerts
  • Agree which applications and projects subscribe, and which teams own each alert route
  • Define the failure conditions that count as alerts, such as failed syncs and degraded health
  • Record the secret handling model for each notification service before any credentials are created
  • Confirm the written scope, deliverables and timeline with your team before work begins

How we build

  • Write triggers and message templates in argocd-notifications-cm for the agreed conditions
  • Create one Kubernetes Secret per service, keeping tokens and keys out of Git and out of templates
  • Add subscription annotations to applications and projects so routing follows ownership
  • Test each trigger against a non-production application and confirm the message reaches the intended channel
  • Review template content so alerts include the application name, revision and a link back to Argo CD

How we hand over

  • Provide version-controlled manifests for templates, triggers and subscriptions, stored in your Git repository
  • Document how to add a new service, channel or subscription without changing core triggers
  • Walk your operators through the runbook for rotating notification credentials
  • Record test results for each alert path so your team can repeat them after changes
  • Confirm handover criteria in writing against the agreed scope

How we support

  • Ongoing support, upgrades and continuous testing are available as retainers
  • Review notification behavior when you upgrade Argo CD to a new minor or patch release
  • Confirm alert paths after changes to projects, clusters or notification service accounts
  • Ask for a quote based on the written scope, which sets the effort for each service you add

Talk to our team.

Tell us what you're working on, whether it's a deployment, an audit, a security test or a cyber range. You'll speak with an engineer who can help you scope it.

  • 30-minute call: free, with no obligation.
  • NDA on request: we can sign before you share details.
  • Clear next steps: a scope and plan after the call.