Skip to main content
πŸ“˜ Complete PDF

Apache Airflow Certification Practice Test 06

Statement Analysis Questions

Question 1 of 20
βš–οΈ Statement Analysis
Question 101
Evaluate the following statements regarding Airflow Directed Acyclic Graphs (DAGs): Statement 1: A DAG defines the workflow structure, execution schedule, and dependency graph between tasks. Statement 2: A DAG stores the actual analytical data generated by task executions directly inside the DAG definition file.
βœ“
A. Only Statement 1 is correct
B. Only Statement 2 is correct
C. Both Statements are correct
D. Neither Statement is correct
Answer
βœ… A. Only Statement 1 is correct
Explanation

1. Core Concept Overview In Apache Airflow, a DAG (Directed Acyclic Graph) is strictly a workflow orchestration definition. It declares tasks, execution order, schedule intervals, and dependencies as code. Airflow DAG files do NOT store analytical datasets.

2. Step-by-Step Technical Breakdown - **Statement 1 is Correct:** A DAG defines how tasks are related, their upstream/downstream execution order, scheduling intervals, retries, and timeouts. It is a mathematical directed acyclic graph where nodes represent tasks and edges represent dependencies. - **Statement 2 is Incorrect:** DAG definition files (`.py`) contain orchestration code, not data payloads. Data processed by tasks resides in external systems (S3, Snowflake, BigQuery, PostgreSQL). Transient metadata between tasks is passed via XCom, while execution states are recorded in Airflow's metadata database.

3. Why Other Options Are Incorrect - **B is incorrect** because Statement 1 is accurate and Statement 2 is false. - **C is incorrect** because Statement 2 violates the fundamental separation of orchestration and data storage. - **D is incorrect** because Statement 1 accurately characterizes the core definition of an Airflow DAG.

4. Architecture & Platform Execution Airflow separates orchestration logic from data storage. The Scheduler continuously parses DAG definition files to generate DAG runs and TaskInstances in the metadata database (PostgreSQL/MySQL). Tasks execute on worker nodes, fetching from and writing to external storage systems (cloud object storage, data warehouses). Storing data inside DAG scripts would cause severe file-parsing latency and crash the scheduler.

5. Practical Code Example

from airflow import DAG
from airflow.operators.bash import BashOperator
from datetime import datetime, timedelta

default_args = {
    'owner': 'data_engineering',
    'retries': 2,
    'retry_delay': timedelta(minutes=5),
}

# DAG defines workflow structure and orchestration metadata only
with DAG(
    dag_id='daily_sales_ingestion',
    default_args=default_args,
    start_date=datetime(2025, 1, 1),
    schedule_interval='@daily',
    catchup=False,
    tags=['retail', 'gold']
) as dag:

    extract_task = BashOperator(
        task_id='extract_pos_data',
        bash_command='python /opt/scripts/extract.py'
    )

    validate_task = BashOperator(
        task_id='validate_records',
        bash_command='python /opt/scripts/validate.py'
    )

    extract_task >> validate_task

6. Key Takeaway Airflow is an orchestrator, not a data storage system. DAG files declare relationships and schedules; analytical payloads must remain in external databases and object stores.

7. Best Practice & Production Recommendation Keep DAG files lightweight and declarative. Never embed data extraction payloads, heavy dataframes, or long-running computations directly inside top-level DAG script code.

8. Real-World Engineering Scenario A junior engineer attempted to load a 500MB CSV file into a global variable inside a DAG Python file so tasks could reference it. The Airflow Scheduler parsing loop froze because it evaluates all DAG files every 30 seconds, causing scheduler heartbeats to fail across the cluster. Moving the data read to an external S3 bucket resolved the outage.

9. Common Exam Trap / Anti-Pattern / Misconception Confusing Airflow task parameters with data storage. DAG files are parsed repeatedly in memory; top-level data reads will cripple scheduler performance.

10. Related InsightfulSaga Tutorial Master these concepts in [Airflow Architecture & Operations](/tutorials/airflow/airflow-dags).

Answered: 0 / 20
Question Navigator (Test 06)0% Answered