Migrating to Apache Airflow

Moving schedules and pipelines into self-hosted Airflow from cron, retired Hadoop schedulers, enterprise job schedulers and managed Airflow services, with coexistence and cutover planned.

Migrate cron jobs to Apache Airflow

Replacing crontabs spread across servers with Airflow DAGs: inventorying jobs, translating schedules to Airflow 3 timetables, adding retries, dependencies and alerting, handling scripts that need local files, and retiring cron host by host.

Migrate from Apache Oozie to Apache Airflow

Leaving Oozie after its retirement to the Apache Attic: mapping workflows, coordinators and bundles to DAGs, timetables and assets, replacing Hadoop actions with Spark, Hive and SSH operators, and decommissioning the Oozie server.

Migrate from Luigi to Apache Airflow

Moving Luigi task pipelines to Airflow: Luigi targets and requires versus Airflow dependencies and assets, scheduling that Luigi leaves to cron, central history and retries, and porting tasks incrementally.

Migrate from Control-M to Apache Airflow

Moving data pipelines off BMC Control-M while keeping enterprise jobs running: the Control-M for Apache Airflow plug-in for a coexistence phase, translating calendars and dependencies, SLA monitoring, and the order to move job folders.

Moving from Amazon MWAA to self-hosted Apache Airflow

Taking DAGs from Amazon Managed Workflows for Apache Airflow to an Airflow you run: matching versions and constraints, scheduler and worker limits you outgrow, multi-team needs, AWS connections and secrets, and a parallel run before cutover.

Migrate AWS Step Functions workflows to Apache Airflow

When data workflows built as Step Functions state machines should move to Airflow: mapping states to tasks, keeping Lambda and Glue steps, running Step Functions from Airflow during the transition, and which workflows should stay event-driven in AWS.

Talk to our team.

Tell us what you're working on, whether it's a deployment, an audit, a security test or a cyber range. You'll speak with an engineer who can help you scope it.

  • 30-minute call: free, with no obligation.
  • NDA on request: we can sign before you share details.
  • Clear next steps: a scope and plan after the call.