Dagster vs Kestra: which to choose for self-hosted pipelines

Dagster or Kestra: when each one fits a self-hosted data platform

Choose Dagster when your pipelines are written in Python by data engineers who think in assets: tables, models and files that each have a definition, dependencies and a freshness question. Dagster's open-source core is Apache-2.0 licensed and organizes orchestration around these assets, so the asset graph, lineage and materialization history are central to how the platform is used. Choose Kestra when pipeline authors span several languages or include people who prefer declarative YAML or a no-code editor, when flows must start from many kinds of events, and when a flow may call scripts, APIs and databases through a large plugin catalog. Kestra's open-source edition is Apache-2.0 licensed, and its YAML, no-code editor and code stay in sync, so a platform team can review flows as Git-tracked files without forcing one language on every author.

The decision usually turns on four questions. First, who authors pipelines day to day: if the answer is a Python data platform team, Dagster's programming model fits; if it is a mix of application engineers, analysts and operators, Kestra's YAML and editor options may reduce friction. Second, how much you need lineage and an asset view: Dagster builds this into the model, while Kestra's event-driven flows are organized around triggers and tasks. Third, identity and audit in the open-source tier: Dagster's pricing page lists RBAC, SAML SSO, catalog search, column lineage, branch deployments, alerts and audit logs as Dagster+ features, while Kestra's edition comparison says the open-source edition has basic authentication only and no audit logging. Confirm with each vendor whether any of these controls are available to you without a paid tier. Fourth, which paid features your controls require, and whether they run in your infrastructure or in a vendor-hosted service.

Before you shortlist either tool, write down the controls your auditors and security team expect for orchestration, such as named access roles, SSO, secrets handling, and retained run history. Map each one to Dagster+ or Kestra Enterprise Edition, then ask both vendors for a written quote against that list. Also weigh the acquisition news: Dagster Labs announced on July 13, 2026 that Prefect is acquiring it, and the company did not say whether the deal had closed. Ask for the roadmap, support terms and any change to Dagster+ hosting in writing before you commit.

Which orchestrator fits which team scenario

This table maps common team scenarios to the orchestrator that usually fits better, and names the condition that makes each choice hold. Read each row as a starting point, then check the conditions against your own skills, identity requirements and hosting model. The main takeaway is that Python-first data platform teams lean toward Dagster, while mixed teams, event-driven operations and governance-heavy rollouts need closer review of what sits in the open-source edition and what sits in paid tiers.

Team scenario	Better fit	Condition that makes it so
Python data platform team building asset graphs	Dagster	Engineers want pipelines declared as Python functions that produce tables or models, with lineage tied to those assets.
Mixed team where analysts write pipelines in YAML	Kestra	Flows are declarative YAML, and a no-code editor and YAML editor stay in sync, so non-Python authors can contribute directly.
Event-driven infrastructure or operations workflows	Kestra	Scheduled and event-driven triggers are a core part of the model, and most automation steps run as tasks in the flow.
AI or RAG workflows exposed to agents	Kestra (confirm fit)	Kestra lists AI agent and RAG task types and a built-in MCP server; confirm the exact agent behavior your workloads need in a proof of concept.
Team needing SSO, SCIM and audit logs on day one	Kestra Enterprise Edition or Dagster+	The open-source Kestra edition has basic authentication and no audit logging, so day-one controls point to a paid tier on either side; confirm the current scope of each in writing before you commit.
Team that needs asset lineage in a catalog from the start	Dagster	Dagster's asset model produces lineage natively, while Kestra flows are task graphs, so lineage depends on what you wire into a catalog; confirm column lineage tier requirements on Dagster+.

How Dagster and Kestra are built and what that means for operations

Dagster and Kestra both run as self-hosted services, but they split responsibilities differently and scale differently. Dagster's open-source deployment is a set of cooperating processes with one daemon and one-replica code location servers, while Kestra's open-source edition defaults to a single server that becomes a high availability design only in the Enterprise Edition. The cards below set out those components and what each one means for the people who run them.

Dagster webserver and replicas

The open-source Dagster webserver serves the UI and GraphQL API and can run with multiple replicas, so user-facing load can be spread across instances. Replicas do not change who runs schedules, which is the daemon's job. Check that your run and event log storage is shared and reachable from every replica before you scale the webserver out, because a SQLite file on local disk will not serve several instances.

One daemon for schedules and queuing

Dagster runs a single daemon that operates schedules, sensors and run queuing. Because there is one daemon, it is a single point to monitor and restart, and a stalled daemon stops scheduled runs even while the UI stays up. Ask your platform team how they will alert on daemon health and what recovery looks like on your orchestrator platform, whether that is Kubernetes with Helm, Docker or Amazon ECS.

Code location servers and their limits

Each code location server, which loads your Python asset definitions, is limited to one replica in open-source Dagster. Teams with large or many code locations should plan for that ceiling and for what a restart of one location means for the assets it serves. Map which code locations own which assets, and confirm the current replica behavior against the deployment documentation before you size the cluster.

Storage choice: SQLite or PostgreSQL

Dagster keeps run, event log and schedule data in storage that defaults to SQLite and can use PostgreSQL. SQLite suits a single-node evaluation, while PostgreSQL is the option that lets the webserver and daemon share state across hosts. Decide early, because moving run history between backends is a migration you will need to plan, test and back up in your own environment.

Swappable launcher, coordinator and executor

Dagster's run launcher, run coordinator and executor are separate components that can be replaced. This lets you choose where runs execute, such as in containers on your own cluster, but it also means the combination you pick becomes part of your support and upgrade surface. Document the chosen components, test each upgrade against them, and confirm with your vendor or DeployOpen scope which combinations your team will own.

Kestra single-server default

Kestra's open-source edition is fully functional for production, but its default deployment runs on a single server, and secret and access management are manual. Operations therefore start small: one host, one set of backups, and a clear procedure for restoring it. The trade-off is that a single-node failure stops flows until the host is recovered, so define your recovery objectives before you accept that model.

Kestra Enterprise high availability

Kestra Enterprise Edition adds a high availability architecture using Kafka with Elasticsearch, Redis, AMQP or GCP Pub/Sub queue backends, along with worker groups and task runners for Kubernetes and cloud batch services. This is a larger set of moving parts to run, secure and upgrade. Ask Kestra which queue backend fits your environment, and budget for operating that messaging layer as well as the orchestrator itself.

Capability comparison: lineage, triggers, identity and extensibility

This table compares how Dagster and Kestra handle authoring, lineage, triggers, Git, plugins, AI features and identity controls. Read the Dagster+ and Kestra Enterprise Edition rows first, because several identity and governance features sit behind a paid tier on each side. Where a cell says to confirm, check the feature against the current vendor documentation before you rely on it.

Capability	Dagster (open source)	Kestra (open source)
Authoring model	Python functions declare software-defined assets, and the asset graph is derived from them	Declarative YAML flows, editable in a YAML editor, a no-code editor or an AI Copilot that stay in sync
Language support	Python-first, with pipeline logic written as Python code and tested like application code	Language-agnostic, with scripts in any language run as tasks inside a flow
Lineage and asset view	Asset-level lineage and an asset graph are built in; column lineage is a Dagster+ feature	Flow-level execution view; the supplied facts do not describe asset-level lineage, so confirm it with Kestra
Scheduled and event-driven triggers	Schedules and sensors run through the single open-source daemon	Scheduled and event-driven triggers are listed in the README as core flow features
Git integration	Code lives in version-controlled Python projects and deploys through code locations	Git version control integration is listed in the README as a core feature
Plugin and integration breadth	Integrations are Python libraries you install and maintain in your own code	Hundreds of built-in plugins for databases, cloud storage and APIs
AI and MCP features	No built-in MCP server is listed in the supplied facts; confirm before planning around one	AI agents and RAG workflows are task types, and a built-in MCP server exposes flows to AI agents
Open-source authentication	Open-source Dagster is self-hosted, so you add and operate access controls around the webserver	Basic authentication only in the open-source edition
RBAC and SSO	Role-based access control and SAML SSO are Dagster+ features	RBAC and SSO are Kestra Enterprise Edition features
Audit logs and secrets management	Audit logs are a Dagster+ feature; secrets handling is in your own deployment	No audit logging and manual secret management in open source; audit logs and external secrets managers such as HashiCorp Vault are Enterprise Edition features

What is free and what is paid in each platform

Dagster is licensed under Apache-2.0, so the open-source project can be run, modified and redistributed under that license without a commercial license from Dagster Labs. The open-source project describes itself as a cloud-native data pipeline orchestrator with integrated lineage and observability, a declarative programming model and testability. The license covers the code, but it does not decide which operational features you get for free, which depends on the edition you deploy.

License terms: Apache-2.0 covers the open-source Dagster codebase you self-host.
Core orchestration: Assets, schedules, sensors and the webserver are part of the open-source deployment.
Paid cloud: Dagster+ is a separate paid service from Dagster Labs and is not the same thing as the open-source project.
Release cadence: Dagster 1.13.26 was released on October 9, 2026, after 1.13.25 on October 1 and 1.13.24 on September 21, 2026.
Acquisition: Dagster Labs announced on July 13, 2026 that Prefect is acquiring Dagster Labs, and it did not say whether the deal had closed.

Confirm with Dagster Labs which parts of the roadmap and support terms apply to the open-source project if the acquisition closes.",

What Prefect's agreement to acquire Dagster Labs means for teams choosing Dagster

On July 13, 2026, Dagster Labs announced that Prefect is acquiring the company. The announcement said Dagster+ continues under its brand and the open-source project will keep being actively developed, but it did not say whether the deal has closed. The cards below cover what is known, what to ask, and when the answers matter most.

What the announcement says

Dagster Labs announced on July 13, 2026 that Prefect is acquiring it. The company said Dagster+ continues under the Dagster+ brand and that the open-source Dagster project will be actively developed. The announcement did not state whether the deal has closed, so treat the transaction as pending until the vendor confirms otherwise. Read the original post directly before relying on any summary, including this one.

Open-source roadmap commitments

The vendor's statement that the open-source project will be actively developed is a stated intent, not a published roadmap. Ask Dagster Labs and Prefect for the planned open-source milestones, the maintainers who will own the core releases, and how the project will be governed after any close. Compare that answer against the current release cadence on GitHub, where Dagster 1.13.26 shipped on October 9, 2026, to judge whether the pace holds.

Dagster+ terms and feature scope

Dagster+ carries the features that many governed teams need, including role-based access control, SAML SSO, catalog search, column lineage, branch deployments, alerts and audit logs. Ask whether those features, their tiers and their pricing will change after the transaction, and whether any current feature is planned to move between tiers. Get the answer in writing, because a Dagster+ feature that a Kestra comparison counts as included may be a paid item that changes later.

Contract continuity and renewal terms

If you sign or renew a Dagster+ contract during the pending period, ask who the contracting entity will be after any close, whether the terms survive a change of control, and what notice you get before prices, support levels or data residency options change. Confirm which party will support deployments that run your code on your own infrastructure. Keep a written exit path, including data export from the run and event log storage, so a change of owner does not lock your history inside a vendor.

Hybrid and self-hosted exposure

A self-hosted open-source Dagster deployment depends on the code and the release cadence of the project, not on the Dagster+ control plane. Teams that run open-source Dagster on their own Kubernetes, Docker or Amazon ECS footprint face less direct exposure to a Dagster+ change, but they still depend on the project's continued maintenance. Ask which parts of your planned stack would need Dagster+ and which would keep working without it, then document that boundary before you commit.

Where this matters most

The acquisition matters most for a team that plans to buy Dagster+ for authentication, audit logs or branch deployments, and that has to justify a multi-year commitment to auditors or a procurement board. It matters less for a team that runs open-source Dagster on its own infrastructure and can absorb a change in vendor. If you are also evaluating Kestra, whose Enterprise Edition carries SSO, audit logs and high availability under its own terms, compare both vendors' written commitments on the same questions.

How to move pipelines from Dagster to Kestra, or back

The sequence turns an inventory of Dagster assets and jobs into a Kestra flow set that has been run side by side with the original and then switched over. Each step has a written check, so the cutover decision rests on evidence rather than on the migration plan.

Inventory assets, jobs, schedules and sensors
The data platform lead and the owners of each code location list every software-defined asset, job, schedule and sensor in the Dagster project, along with its upstream and downstream dependencies. Record which assets write to which tables or buckets, which ones depend on Python libraries that have no Kestra plugin equivalent, and which ones run on a schedule versus on an event. The inventory is finished when every asset and trigger has a row, an owner and a stated target: transfer as code, rebuild as a flow, or retire.
Classify what transfers as code and what gets rebuilt
Engineers sort the inventory into two groups. Logic that is already plain Python or SQL can often move into a Kestra script task or a SQL task, since Kestra's README says scripts can run in any language. Dagster asset graphs, partitions and asset-level lineage do not map one to one onto Kestra flows, so those are rebuilt as YAML flows with explicit task dependencies. Confirm each group against the Kestra documentation for the task types you need, because the plugin set changes between releases. The classification is complete when each item has a decision and a named reviewer.
Plan the Kestra 2.0 changes before writing flows
If the target is Kestra 2.0 or later, the platform team reads the 2.0 migration guide referenced in the Kestra README before writing any flow. That guide documents breaking changes, including replacing the ForEach and ForEachItem tasks with a single Loop task. Anyone copying flow patterns from older examples should check them against the 2.0 syntax. Note that Kestra v2.0.6 was published on October 13, 2026, and that the 1.3 line still receives patch releases, so the team should pin the version it tests against. The step is done when the target version is pinned and the breaking-change list is applied to the inventory.
Port triggers, secrets and access controls
The platform team re-creates each schedule and event trigger as a Kestra trigger and maps every secret the Dagster code reads from environment variables or its own secret handling to the Kestra mechanism in use. If the team needs an external secrets manager such as HashiCorp Vault or Azure Key Vault, that is a Kestra Enterprise Edition feature per Kestra's edition comparison, so confirm the licensing before the design is final. The open-source edition has basic authentication, manual secret management and no audit logging, so access rules and audit requirements from the Dagster side must be written down and checked against the chosen edition. The step closes when a trigger test fires each ported schedule and event in a non-production namespace, and the secret inventory has no unresolved entries.
Rebuild lineage and observability checks
Dagster's asset view gives lineage and run history without extra work, while Kestra's execution history is organized around flows and tasks. The data team defines the checks that replace the asset lineage view for the auditors and downstream owners who depend on it, such as a table of run outcomes, row count checks or a catalog link. Dagster's catalog search, column lineage and alerts are Dagster+ features per the pricing page, so if those were used in production, confirm which equivalent exists in your chosen setup. The step is complete when every asset with a downstream consumer has a documented check that a named person has reviewed.
Run both pipelines in parallel and compare outputs
The team runs the Dagster and Kestra versions of each pipeline over the same inputs for a period it sets in advance, and compares the written outputs, run outcomes and failure handling. Sources are read from the same snapshot where possible so that differences can be traced to the code rather than to timing. Any difference is logged with a cause and a decision. Parallel running also tests the infrastructure: the self-hosted Dagster components (webserver, daemon and code locations) and the Kestra server, worker and queue setup each need to be checked under load. The step is done when every output difference has an explanation and the team has signed off on the comparison report.
Cut over, then keep a rollback path
Once the parallel run is signed off, the team disables Dagster schedules and sensors for each migrated pipeline in order, starting with the lowest-risk ones, and enables the matching Kestra triggers. Each cutover is confirmed by the first scheduled run on the Kestra side and a check of its output. The Dagster deployment stays in place, read-only where possible, until the team has agreed that rollback is no longer needed. The sequence produces a documented cutover record for each pipeline. If the direction is Kestra back to Dagster, the same steps apply in reverse, with the Dagster asset definitions rebuilt from the Kestra flow inventory.

Where implementation projects with each platform usually run long

On the Dagster side, the effort that most often runs long is asset modeling. Each asset is a Python function that declares a logical unit of data, so the team must decide early which tables, models and files become assets, how dependencies between them are expressed, and how the code location is laid out. Teams that split code locations poorly end up with duplicated dependencies and slow deploys, and open-source Dagster limits each code location server to one replica, which should be part of that layout decision. Storage is the other early choice. Run, event log and schedule storage default to SQLite, and moving to PostgreSQL is a configuration change you should plan and test before the first production run. Ask Dagster or your implementation partner to confirm how the storage, run launcher and executor you select behave under your expected run volume.

On the Kestra side, the work is spread across plugin configuration, secrets and identity. Flows are declarative YAML, so most of the build effort goes into configuring plugins for your databases, cloud storage and APIs, and into deciding how credentials are supplied. The open-source edition provides manual secret and access management and only basic authentication, so teams that need SSO, SCIM, RBAC, audit logs or external secrets managers such as HashiCorp Vault or Azure Key Vault must budget for Kestra Enterprise Edition and its high availability setup, which adds Kafka, Elasticsearch and a queue backend. Identity integration is the step most likely to overrun, because it depends on your identity provider, group mapping and service account design. Confirm with Kestra which of these features your edition and deployment model include before you commit to a timeline.

Across both platforms, three areas overrun most often: identity integration, data quality contracts and version upgrades. Dagster places SAML SSO, RBAC and audit logs in Dagster+, so an open-source deployment will need its own access controls, and data quality expectations must be written into asset checks or flow logic that someone owns after handover. Upgrades are a recurring cost for both. Kestra 2.0 is a major release with breaking changes and a dedicated migration guide, including replacing ForEach and ForEachItem with a single Loop task, while Dagster releases frequently, so plan a test environment and a regression suite for each upgrade. Ask each vendor for its current upgrade and deprecation policy in writing, and ask your implementation partner for a fixed-scope list of which identity, quality and upgrade tasks are included.

Questions buyers ask before choosing Dagster or Kestra

Does Dagster or Kestra suit a team that writes only Python?

Dagster suits a Python-only team most directly, because you declare data assets as Python functions and the framework's programming model, testing and lineage all live in that code. Kestra suits mixed teams better, since its flows are declarative YAML and its task types run scripts in any language. A Python-only team that wants pipelines close to its code and tests will usually find Dagster the more natural fit. A team that also needs analysts, platform staff or non-Python scripts in the same flows should test whether YAML flows reduce friction. Run a short proof of concept with one real pipeline from each side before deciding.

Which Dagster+ or Kestra Enterprise features add effort to a deployment?

Several governance features sit outside the open-source editions in both products. Dagster+ lists role-based access control, SAML SSO, catalog search, column lineage, branch deployments, alerts and audit logs as paid features. Kestra Enterprise Edition adds SSO, SCIM and RBAC, service accounts and API tokens, external secrets managers, audit logs, multi-tenancy, worker isolation and a high availability architecture. Effort depends on whether you buy the paid tier or build equivalents yourself. Buying reduces integration work but adds a recurring subscription. Building means you own identity wiring, secret handling, audit retention and high availability testing. Ask any vendor or implementation partner for a written scope that lists each feature, who configures it and what it takes to validate it.

How long does a migration to Dagster or Kestra take, and what drives it?

No supplied figure gives a migration duration, so treat any number you hear as a claim to verify. The main drivers are the number of pipelines, how many depend on custom operators or plugins, the state stored in your current scheduler, and how much lineage or alerting your teams rely on today. Dagster migrations often involve rewriting job logic as assets, which changes how dependencies and tests are expressed. Kestra migrations often involve translating code into YAML flows and checking that plugins cover each external system. Ask for a written inventory of pipelines, a cutover plan with parallel runs, and a rollback path. Fixed-scope work agreed in writing before starting makes these items easier to compare.

Can analysts author Kestra flows without engineers?

Partly, and the answer depends on what analysts need to change. Kestra's README describes a YAML editor, a no-code editor and an AI Copilot that stay in sync, so analysts can build or adjust flows without writing Python. Scripts that run in any language still need someone who can maintain them, and Enterprise features such as RBAC and service accounts determine who may edit production flows. Git version control integration supports review of changes before they reach production. Dagster asks more of its authors, since assets are Python functions. Test the no-code editor on a representative flow, and confirm with Kestra which editing permissions the edition you plan to run provides.

What does running Dagster or Kestra require from the internal team?

Both platforms need people who can operate a service with a database, scheduling, worker capacity and upgrades. For self-hosted Dagster, the webserver can run multiple replicas, a single daemon handles schedules, sensors and run queuing, and code location servers run one replica each, so you need someone who understands that topology. Run storage defaults to SQLite and can use PostgreSQL, which is a decision your operators must make. Kestra's open-source edition defaults to a single server and leaves secret and access management to you, while Enterprise adds Kafka, Elasticsearch and Redis-class components for high availability. Plan for platform engineering, security and on-call roles, and confirm which of these your staff already cover.

Does the Prefect deal change the Dagster decision?

It changes the questions you should ask, though not the core product comparison. Dagster Labs announced on July 13, 2026 that Prefect is acquiring it, said Dagster+ continues under its own brand, and said the open-source project will be actively developed. The announcement did not say whether the deal had closed. Before committing, ask the vendor about roadmap ownership, pricing changes for Dagster+, licensing of the open-source project, and support commitments during integration. Keep an exit plan in view, since assets written in Python are portable only as far as your own code is. Revisit the decision once the deal status and any product changes are published.

What does DeployOpen cover if we choose one of these platforms?

DeployOpen plans, implements, customizes, integrates, hardens, backs up, upgrades, documents and supports self-hosted open-source platforms inside your own infrastructure, which can include Dagster or Kestra. Most engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts. Ongoing support, upgrades and continuous testing are available as retainers. DeployOpen does not publish a price list, so a quote follows from the written scope. Ask for a scope that names the pipelines, identity integration, secrets handling, backup and upgrade responsibilities, and the internal roles that must sign off.

How DeployOpen helps choose, implement or migrate between them

DeployOpen is a Chainscore Labs company that plans, implements, integrates, hardens, backs up, upgrades and documents self-hosted open-source platforms inside your own infrastructure, including your cloud accounts, data centers or air-gapped networks. For a Dagster or Kestra decision, we evaluate both platforms against your pipeline inventory, your identity and secrets requirements and your hosting constraints, and we do not favor either one before that evaluation is done. Our work is delivered under fixed-scope written agreements, and a quote follows from the scope you approve.

Before you commit, the most useful engagement is a scoped evaluation that maps your current jobs to each model. Ask which of your pipelines are asset-shaped, meaning they produce tables or models that other teams consume, and which are event-driven or task-shaped, such as file arrival triggers or API calls. Then confirm which controls you need on day one. Dagster's role-based access control, SAML SSO, catalog search, column lineage, branch deployments, alerts and audit logs are listed as Dagster+ features, while Kestra's basic authentication, manual secret handling and absence of audit logging in the open-source edition map to Kestra Enterprise Edition features such as SSO, SCIM, RBAC, external secrets managers and audit logs. Confirm current edition terms directly with each vendor before you set a budget.

If you are already running one of these tools, DeployOpen can also plan a migration, implement the target deployment and document the cutover. The scope should name the pipelines in each migration wave, how run history and schedules will be carried over, and how the old and new systems will run in parallel. Support and upgrades after go-live are available as retainers, which matter for both platforms because Dagster and Kestra each ship frequent releases and Kestra 2.0 includes breaking changes covered by its migration guide. Ask for the scope, deliverables and timeline in writing before work begins.

Dagster vs Kestra