There is one DAG that I wrote–and it’s one of the first DAGs that I ever put into production–that has tasks that are dynamically allocated. What I mean is that there are a number of very similar tasks that I need to perform as quickly as possible, thus they need to run in parallel. The […]
Tag: Variables
3 Easy Airflow Tests
I haven’t exactly found a wealth of information on how to test with Airflow. Here are 3 tests though that I’ve used with every single DAG that I’ve written. The first one does nothing more than verify that there are no syntax errors within the DAG file, the second one confirms the existence of and […]