Migrate from Fivetran to dlt

Migrate from Fivetran to dlt: what the move involves

A move from Fivetran to dlt replaces each Fivetran connection with a dlt source or pipeline that your team writes and owns in Python, scheduled in an orchestrator you already run. The effort depends less on how many connections you have than on three things: how much custom logic each source needs, whether the downstream dbt models rely on Fivetran-managed behavior such as its column naming, soft-delete flags and sync metadata tables, and how many of your sources have an existing dlt source you can reuse. Count these before you estimate anything, because they decide whether the work is a rebuild of a few pipelines or a set of new connectors.

dlt is a library, not a platform. It installs into existing code and runs in an AWS Lambda function, an Airflow DAG, a notebook or a local machine, as the dlt README describes. That means your team takes over what Fivetran's control plane did for you: scheduling, run state, retries, alerting and fixing a connector when an upstream API changes. dlt handles incremental loading, schema inference and schema evolution inside the pipeline, but the pipeline still needs a home, credentials managed in your secrets store, and someone on call for failures. Confirm with your platform team which orchestrator, secrets store and monitoring stack the pipelines will use before you start rebuilding.

Internal data engineers usually do the rebuild, since they know the sources and the downstream models. A partner is most useful for scoping the connector inventory, setting up the shared pipeline project and orchestration pattern, and running the parallel load and row count reconciliation that decides when Fivetran can be switched off. Ask any partner for a written scope that lists each source, the destination tables and naming it must reproduce, the reconciliation method, and the cutover criteria. Check the dlt version you pin against the dlt project on PyPI and your own test results, not against this page.

Which Fivetran capabilities map cleanly to dlt

This table maps each Fivetran capability a pipeline depends on to its dlt equivalent and marks it as clean, needs rework or gap. Read the status column first, then the notes, because the largest difference is not a feature but who maintains each source. The main takeaway is that dlt covers loading mechanics well, while every connector Fivetran manages becomes code your team owns.

Fivetran capability	dlt equivalent	Status and notes
Pre-built managed connectors	Sources written as Python, such as REST API and SQL database sources	Needs rework. No vendor-maintained connector catalog. Each source is code your team writes, tests and upgrades when an API changes.
Connector count	Library sources and community or dltHub-supplied sources	Gap. The library's source count is not Fivetran parity. Confirm each source you need in the dlt docs before committing.
Source types: databases, files, APIs	SQL databases, REST APIs, files on S3, GCS and Azure, pandas, Polars and Arrow DataFrames	Clean for common shapes. Confirm vendor-specific authentication, pagination and rate limits for each API.
Destinations	More than twenty supported destinations, including Snowflake, BigQuery, Postgres, Databricks, DuckDB and filesystem storage	Clean if your warehouse is listed. Confirm the destination version and the load method you need before migrating.
Schema inference and nested data	Built-in schema inference and nested data normalization	Clean for most JSON-like payloads. Check how deeply nested arrays map to child tables against your downstream dbt models.
Incremental syncs	Incremental loading and merge strategies with state kept between runs	Needs rework. Map each Fivetran sync mode to an explicit cursor or merge key, and plan the initial backfill.
Schema change handling	Schema evolution and schema contracts declared in Python	Needs rework. Contracts must be set deliberately, since Fivetran's automatic handling and dlt's contract rules can behave differently on column changes.
Operations: scheduling and monitoring	Runs in an orchestrator you already operate, such as Airflow, or a serverless function	Gap. Alerting, retries and run history must be built or wired to your existing tools.

Exporting Fivetran configuration and rebuilding sources in dlt

Before writing any dlt code, export what Fivetran knows about your connections, destinations and schemas, and use that export as the inventory of pipelines you must reproduce. The cards below cover what to take out of Fivetran, what to strip before anything is stored in version control, and what has to be rebuilt by hand in dlt and the systems around it.

Connection inventory from the REST API

Use the Fivetran REST API to list every connection, its connector type, sync frequency, paused state and destination group. This list becomes your migration plan: each entry maps to one dlt source, one destination dataset and one scheduled job. Record which connections are paused or unused, because those are candidates for retirement rather than rebuilding. Ask your Fivetran administrator to confirm that the API key you use has read access to every destination group, since a partial export produces a partial plan.

Terraform state as a second inventory

If your team manages Fivetran with the Terraform provider, the state and configuration files are a structured second inventory of connections and destinations. Compare them against the REST API export, because manually created connections often exist only in the Fivetran interface. Treat the Terraform files as documentation of intent, and expect to rewrite the resource definitions for dlt rather than port them line by line.

Stripping credentials from exported configuration

Exported configuration can contain authentication fields, tokens and connection strings for source systems. Remove every secret before committing the export to Git, and store the sanitized version only as a plan. Rotate source credentials that were ever exposed in shared exports or tickets. The export should describe which credential each source needs, not contain the credential itself.

Source credentials in a secrets store

Each dlt source reads its credentials from configuration, environment variables or a secrets provider that your team already operates. Create one secret per source system, assign it to the service account the pipeline uses, and document who can rotate it. Avoid placing credentials in secrets.toml files committed to repositories. Confirm with your security team which secrets manager is the approved store before the first pipeline runs.

Destination datasets and naming

Fivetran creates destination schemas with its own naming conventions, and downstream dbt models and BI reports reference those names. Decide whether dlt writes to the same dataset and table names or to a new dataset that you cut over to later. Set dataset names explicitly in the pipeline rather than relying on defaults, and compare column names and types against the Fivetran output. Changes in naming break models silently, so check them during parallel runs.

Incremental cursors and load state

Fivetran tracks its own sync position for each table, which dlt does not inherit. Choose the cursor column for each incremental resource, such as an updated timestamp or a monotonically increasing ID, and decide on the initial backfill window. Where a source supports deletions, decide how removed rows are handled before the first run. Ask each data owner to confirm the cursor semantics, because a wrong cursor either loses changes or reloads full history on every run.

Downstream dbt and BI dependencies

Map every dbt source, model and BI dataset that reads from a Fivetran destination schema. These dependencies determine whether a switchover is safe, and they are often undocumented. Update the source definitions in dbt to point at the dlt output, then run dbt tests against both schemas. List the owners of each dashboard so they can confirm that figures still reconcile after cutover.

How to sequence the cutover from Fivetran to dlt

This sequence moves each Fivetran connection to a dlt pipeline in stages, so every table is proven against Fivetran's output before Fivetran is switched off. Each step ends with a named artifact that the next step depends on.

Inventory every connection and its consumers
Export the connection, destination and schema configuration from Fivetran as described in the configuration export guide, then list each connection with its source type, destination schema, sync frequency and the downstream models, dashboards and reverse ETL jobs that read from it. Ask the business owner of each consumer to rank the connections by how much they depend on them. The output is an inventory list with a rank and a named owner for every connection, which becomes the migration plan.
Pilot one low-risk source as a dlt pipeline
Choose a low-volume, non-critical connection, such as a REST API or a small SQL database, and rebuild it as a dlt source in the Python project your team already maintains. Declare the primary key, write-disposition and incremental cursor explicitly, and add schema contracts for the columns downstream models depend on. Run the pipeline from the orchestrator you already use, such as Airflow. The output is a working pilot pipeline with unit tests for its state, meaning the stored cursor values advance correctly across two consecutive runs.
Run both pipelines in parallel on separate schemas
Point the dlt pipeline at a destination schema with a distinct name, such as a suffixed schema beside the Fivetran schema, so both loads write without touching each other. Keep Fivetran running on its normal schedule and let the dlt pipeline run at the same cadence. Confirm that the dlt pipeline produces the same table names and column types that the downstream models expect, and record any mismatch for fixing before cutover. The output is two populated schemas for each migrated connection and a list of schema differences to resolve.
Reconcile row counts and keys per table
For each table, compare row counts between the Fivetran schema and the dlt schema at the same point in time, and check that primary keys are unique and that no keys are missing on either side. Compare a sample of recent and historical records column by column, and confirm that soft-deleted or hard-deleted rows are handled the same way in both. Investigate every difference, because timing gaps and deletion handling are the most common causes. The output is a reconciliation report per table with its results and an agreed sign-off from the consumer owner.
Repoint consumers and cut over the connection
Change the dbt sources, views or extract queries that read the Fivetran schema so they read the dlt schema, then run those downstream models and compare their outputs with the values produced before the change. Pause the Fivetran connection once the consumer owner has signed off, and keep it paused rather than deleted for a fixed observation period set by your own change policy. The output is a repointed set of downstream models and a connection that no longer feeds production.
Retire the Fivetran connection with its final snapshot
Before deleting a connection, export its final configuration and keep a copy of its schema definitions and sync history in your records, so the migration can be audited later. Remove the Fivetran connection, then remove the old schema only after the observation period has ended and no consumer reads it. Update the pipeline inventory so each source names its dlt owner, schedule and contract. The output is a retired Fivetran connection with a stored final snapshot and an updated inventory that shows only dlt pipelines in production.

How the migration changes by connector count and starting point

A small estate of a handful of API and database connectors feeding a single warehouse is the simplest case for dlt. Each source becomes a Python module with its own resources, and the destination is one set of credentials and one dataset. The pilot can cover every connector in the first iteration because there are few enough to review line by line. Most of the effort goes into the source code itself: authentication, pagination, and the cursor fields that drive incremental loads. The parallel run can be short, but it still has to span enough load cycles to show that incremental state behaves correctly across a full business cycle.

Pilot scope: Migrate all connectors in one pass, starting with the least business-critical one to learn the patterns.
Schema settings: Recreate primary keys, write dispositions and nested-data handling by hand, comparing each table against the Fivetran output.
Orchestration: A single scheduled job, such as one cron entry or one Airflow DAG, is often enough.
Parallel run: Run both systems against the same warehouse schema under different names, and let the team choose the length based on how many load cycles it needs to see a full set of changes, including deletes.
Row counts: Reconcile each table after every cycle and keep the comparison queries in version control.

Before cutover, confirm that every source table has a documented incremental cursor and that deleted records are handled the same way in both systems.

What drives the cost and duration of a dlt migration

The largest one-time cost is engineering time to write and test each dlt source, because dlt is a library and not a managed service: someone on your team writes the extraction code, declares the resource and its primary key, and sets the write disposition for every table Fivetran currently syncs. Each connection you rebuild adds a build, test and review cycle, so the number of Fivetran connections and how many of them use custom or unusual APIs will shape the effort more than any other factor. Connectors that map to a dlt REST API source or a standard SQL database source are usually the quickest to move. Reconciliation is a second one-time cost: comparing row counts and key-level values between the Fivetran tables and the dlt tables over the parallel run, and then decommissioning the Fivetran connections, the destination credentials and any downstream references. Ask for an itemized estimate per connection, not a single figure for the whole migration.

Recurring costs come from three places. The first is run cost: compute for the orchestrator workers (for example, an Airflow deployment you already operate or a serverless function) and warehouse load costs for the destination, which you will pay twice during the parallel run because Fivetran and dlt both write to the same warehouse. Compare the warehouse line items on your cloud or warehouse invoice before and after cutover. The second is monitoring and on-call time: dlt does not alert you on your behalf, so you need failure notifications, freshness checks and a named owner for each pipeline. The third is connector maintenance, since source APIs change, and each change to a dlt source is code you own. Plan for upgrades of the dlt library itself, and pin the version you tested. Your Fivetran invoice, broken down by connection and by monthly active rows, is the baseline for deciding whether these recurring costs are lower.

What makes a dlt migration longer or more expensive

This table lists the factors that stretch or raise the cost of moving Fivetran connections to dlt pipelines. Read each row as a driver: the first column names it, the middle column explains why it matters, and the last column gives the step that keeps it under control. The main takeaway is that connector count matters less than hidden Fivetran behavior, cursor logic and reconciliation requirements.

Driver	Why it changes effort	How to keep it down
Number and type of sources	Each Fivetran connector becomes code you own, and API sources with pagination, rate limits and auth quirks take longer than database sources.	Start with a small set of stable, well-documented sources and build a template before porting the rest.
Custom transformations hidden in Fivetran-managed behavior	Fivetran normalizes nested data, renames columns and applies type mapping by default, so downstream models may depend on behavior nobody wrote down.	Export the destination schema from Fivetran and diff it against dlt output to find implicit transformations.
Incremental cursor complexity	Replication keys that are not monotonic, soft deletes and late-arriving records require explicit merge keys and state handling in dlt.	Document each table's cursor and deletion logic, then test incremental runs against a sample of changed records.
Schema drift frequency	Sources that add or change fields often force schema contract changes and code updates, which recur for as long as you run the pipeline.	Set schema contracts deliberately per table and review drift alerts on a fixed cadence.
Existing orchestrator maturity	Teams with Airflow or another scheduler already in place need little new infrastructure, while teams starting from scratch must add scheduling, secrets and alerting.	Reuse the scheduler, secrets store and alerting you already run instead of building new ones for dlt.
Reconciliation rigor required by finance or compliance	Auditors or finance teams may require row counts, checksums and sign-off per table, which adds test design and parallel run time before Fivetran is switched off.	Write row-count tests early, run both pipelines in parallel for the agreed period, and record results per table.

What usually goes wrong when moving to dlt

Most failed Fivetran-to-dlt moves fail on state, schemas and ownership rather than on Python code. The cards below cover the six risks we see most often, each paired with a check to run before Fivetran is switched off.

Incremental state reset during cutover

dlt tracks incremental cursors in pipeline state stored at the destination, and a new pipeline name or a fresh working directory starts that state from zero. The first run can then duplicate rows that already exist or leave gaps where Fivetran's cursor ended. Before the first production run, confirm the pipeline name, the dataset, and the initial value for each incremental resource, and check that the cursor matches the last row Fivetran loaded.

Silent schema changes breaking dbt

dlt infers schemas and evolves them when a source adds columns or changes types, which is useful but can rename or retype columns that downstream dbt models select by name. Fivetran's table and column naming will not match dlt's normalized names automatically. Pin the expected column names in a schema contract, compare the generated tables with the Fivetran output, and run dbt tests against the new tables before pointing models at them.

Credentials and rate limits handled differently

Fivetran manages OAuth refresh, retries and API quota behavior inside its connector. In dlt, those responsibilities move into your code and your secret store, so an expired token or a throttled API can stop a run without the retry behavior you had before. Document where each credential lives, how it rotates, and what the source's rate limit is, then test retry and backoff settings against the API in a non-production environment.

No alerting on failed runs

Fivetran shows sync status and failure notices in its interface, and that visibility disappears once pipelines run inside your scheduler. A failed dlt load in an Airflow DAG or a serverless function can go unnoticed for days while dashboards show stale data. Configure failure notifications in the orchestrator, add freshness checks on the destination tables, and decide who receives the alert before Fivetran is paused.

Connectors with no owner

A dlt source is code your team maintains. When a SaaS vendor changes an endpoint, deprecates a field or alters pagination, nobody is paid to fix the connector, and the pipeline breaks on the next schedule. Before cutover, name an owner for each source, record the API version it targets, and agree on how breaking changes reach that owner. Ask whether the source already exists in the dlt verified sources catalog so that some maintenance is shared upstream.

Double counting during parallel runs

Running Fivetran and dlt into the same destination tables during the overlap window inflates row counts, and merge keys that were never validated can hide the duplication. Load dlt into a separate dataset, reconcile counts and checksums for key tables against Fivetran's output each day, and only point production models at the shared tables after the counts match for an agreed number of cycles.

Questions teams ask before moving from Fivetran to dlt

How much effort does moving from Fivetran to dlt take?

The effort depends mostly on how many Fivetran connections you run, how many of them need custom logic, and how complicated your downstream models are. Each connection needs a dlt source, a destination, a write disposition and an incremental cursor or merge key, and each one needs its own reconciliation before cutover. Connections that map to a documented dlt source with simple tables move quickly, while sources with nested payloads, unusual pagination or custom authentication take longer. Ask your implementation partner to split the work by connection so you can see which items drive the estimate.

Can a small data team move from Fivetran to dlt without outside help?

A small team with solid Python skills and someone who already runs scheduled jobs can often start alone, especially with a few well-understood sources. The harder part is operating the pipelines over time: monitoring failed loads, handling API changes, rotating credentials and keeping schema changes under control. If nobody on the team has owned production Python jobs before, a fixed-scope engagement can cover the first pipelines, runbooks and handover, which gives your team a working reference before you take on the full set.

What skills and infrastructure does a dlt migration require?

You need engineers comfortable writing and testing Python, an orchestrator you already trust, such as Airflow, and credentials with read access to each source and write access to the destination. dlt is a library rather than a platform, so it runs inside your existing code, which means you also need a place to store secrets, a version-controlled repository and a way to run tests. Check that your warehouse account has the roles needed to create and alter tables in the staging and production schemas before the project begins.

How long should the parallel run last, and who decides when to switch off Fivetran?

The length of the parallel run depends on your reporting cycles and on how many downstream consumers depend on each table. Run both systems until every table has matched through at least one full business cycle that includes month-end or any other closing process you care about, and until stakeholders who own those reports sign off. Name the decision owner in the plan, usually the data platform lead with sign-off from finance or analytics owners, and write the acceptance criteria down before the run begins. Do not set a fixed number of days in advance without checking your own cycles.

Does dlt cover every connector that Fivetran offers?

No one should assume full coverage. dlt has sources for REST APIs, SQL databases and files in local or cloud storage, and it supports more than twenty destinations, but a Fivetran connector for a specific SaaS application may not have a matching dlt source or may need a custom one. Confirm each source individually in the dlt documentation, and list any gaps before you commit to a timeline. Custom sources are normal work for a dlt project, but they change the cost, so treat each one as a line item in the written scope.

What happens to historical data and initial loads when we switch to dlt?

Historical data is the part most teams underestimate. Fivetran has already loaded years of history into your destination, and dlt starts from the state you give it, so you need to decide whether to keep existing tables and continue from a cursor, backfill from the source, or rebuild from scratch. Initial loads can be large and slow, and API rate limits or source retention windows may stop you from re-extracting old data at all. Confirm the retention window of each source before you plan the backfill, and keep the Fivetran tables untouched until reconciliation is complete.

What does a fixed-scope DeployOpen engagement cover for a dlt migration?

A fixed-scope engagement covers the agreed list of connections, the dlt sources and destination configuration you approve, scheduling in your existing orchestrator, row-count and schema reconciliation against Fivetran output, and the runbooks your team needs to operate the pipelines. Scope, deliverables and timelines are agreed in writing before work starts, and a quote follows from that written scope. Ongoing support, upgrades and continuous testing are available as separate retainers. DeployOpen prepares the evidence and controls for your review but does not issue audit reports or certifications.

How DeployOpen scopes and runs a Fivetran to dlt migration

DeployOpen is a Chainscore Labs company that plans, implements, hardens, documents and supports self-hosted open-source platforms inside your own infrastructure. For a Fivetran to dlt move, we turn your existing connections into a written, fixed-scope plan, build the dlt pipelines in your environment, and reconcile row counts before anyone switches off Fivetran.

How we scope

We inventory every Fivetran connection, destination, schema setting and sync schedule that the written scope covers.
We agree in writing which sources move to dlt, which stay on Fivetran for now, and which need a decision first.
We confirm where pipelines will run, such as your existing Airflow, serverless functions or notebooks, before work starts.

How we build

We write dlt sources and resources in Python, recreating incremental cursors, merge keys and schema contracts from the agreed inventory.
We wire scheduled runs into the orchestrator you already operate, with secrets kept in your own secrets store.
We run dlt pipelines in parallel with Fivetran and test row counts, key uniqueness and column types against the destination.

How we hand over

We document each pipeline, its state location, its schema contract and the reconciliation results that justified cutover.
We provide a runbook covering reruns, backfills, failed loads and how to add a new source to the dlt codebase.
We walk your data engineers through the repository, tests and deployment steps so the team can maintain pipelines alone.

How we support

Ongoing support, upgrades to newer dlt releases and continuous testing of pipelines are available as a retainer.
Retainer scope, response expectations and review cadence are agreed in writing, with no published price list.
Any quote follows from the written scope, so ask us to itemize the source count, destinations and orchestration work.

Migrate from Fivetran to dlt