Airflow

Apache Airflow 2.10.5 on AlmaLinux 9 with Free Maintenance Support by Pcloudhosting

Apache Airflow is an open-source workflow orchestration platform used to programmatically author, schedule, and monitor batch-oriented data and infrastructure workflows. This AZURE Marketplace image provides a production-ready Airflow deployment packaged on AlmaLinux 9 and engineered for reliability, extensibility, and operational control.

Workflows as Code

All workflows in Apache Airflow are defined using Python-based DAGs, enabling:

• Dynamic DAG generation
• Parameterized and reusable pipelines
• Version control through Git
• Unit and integration testing of workflows

Extensible Framework

Airflow is designed as an extensible orchestration framework. Developers can:

• Create custom operators, sensors, and hooks
• Develop executors and plugins
• Integrate with internal platforms and external services
• Maintain explicit and readable execution logic

Airflow favors explicit dependency definitions and execution semantics. Native Jinja templating is supported for runtime configuration and parameter substitution.

Key Platform Components

• Python-native workflow development
• Web-based user interface for monitoring and control
• Rich ecosystem of integrations
• Flexible scheduling and execution semantics
• Scalable deployment architecture

Supported Runtime Environment

• Python: 3.10 – 3.13
• Databases: PostgreSQL 13+, MySQL 8.0+, SQLite (development/testing)
• Kubernetes: Supported for executor-based scaling
• Memory: Minimum 4 GB RAM recommended (scales with workload)

AZURE-Optimized Deployment Model

• Preinstalled Apache Airflow core components
• Secure and protected default configuration
• Optimized for AZURE Virtual Machine deployments
• Compatible with load balancers, external databases, and object storage

What Airflow Is (and Is Not)

Ideal Use Cases

• Batch data pipelines
• ETL and ELT orchestration
• Machine learning training and evaluation workflows
• Infrastructure and operations automation
• Scheduled operational and maintenance jobs

Not Intended For

• Long-running streaming workloads
• Real-time event processing

Streaming and real-time workloads are typically handled by dedicated streaming platforms, while Airflow coordinates batch and orchestration logic.

Licensing & Support

Apache Airflow is open-source software distributed under the Apache License. This AZURE Marketplace image is independently packaged, maintained, and supported by Pcloudhosting. Optional paid support plans are available. No warranty, express or implied, is provided.