Strengthening Security for Apache Airflow

Written by Jarek Potiuk and Pierre Jeambrun

The Sovereign Tech Fund supports the development, improvement, and maintenance of open digital infrastructure. Its goal is to sustainably strengthen the open source ecosystem, focusing on security, resilience, technological diversity, and the people behind the code.

The Sovereign Tech Fund Contribute Back Challenges sparked a compelling idea for four individual contributors deeply committed to security within the Apache Airflow project. These contributors saw an opportunity to apply to the Securing FOSS Software Production Challenge and reached out to the Sovereign Tech Fund to present a comprehensive set of enhancements to apply to Airflow.

Apache Airflow is an open source platform designed for the development, scheduling, and monitoring of batch-oriented workflows. The user-friendly web interface facilitates efficient management of the state of workflows. Airflow is deployable in many ways, varying from a single process on a laptop to a distributed setup capable of handling the most substantial workflows.

Because Airflow serves as the orchestrator for tens or hundreds of external services and tools,  the workflows it executes have access to all of them, effectively acting as a gateway. In the event of a security breach within Airflow, this gateway status could be exploited. So, ensuring Airflow’s security is paramount for protecting data pipelines. 

In the proposed improvements, the Airflow volunteer contributors aim to strengthen the tooling and security processes governing Airflow. This includes features like implementing a Software Bill Of Materials, ensuring swift response and resolution to security vulnerability reports, integrating static code analysis to preempt vulnerabilities, and enhancing and clarifying Airflow’s security model for the benefit of users and security researchers. The proposal also includes a focus on isolating component access to a shared database to bolster overall system security and delivering security patches faster to the end user by improving the release process.

The application has received approval from the Sovereign Tech Fund, providing the four individual contributors the resources to concentrate on these specific areas of the project with heightened attention and priority. 

The Airflow project would like to thank the Sovereign Tech Fund for their contributions to improve open source security.