Skip to main content
DatabricksPySparkSnowflakeApache Airflow

Operators Basics in Apache Airflow

In the previous chapter, you created your first DAG — a blueprint of execution.

But a blueprint alone doesn’t do anything.

Now it’s time to answer the most important question:

Who actually does the work inside a DAG?

The answer is simple and powerful:

Operators​

What is an Operator in Airflow?​

An Operator defines a single unit of work in Airflow.

Think of operators as workers on an assembly line:

  • One runs a Python function
  • One executes a shell command
  • One simply marks a step as complete

📌 Key Rule:

One operator = one task

Real-World Story: Operators as Team Members​

Imagine a data pipeline team:

RoleAirflow Operator
Engineer running Python logicPythonOperator
System admin running scriptsBashOperator
Project manager checkpointEmptyOperator

Each has a clear responsibility.
That’s how clean Airflow pipelines are built.

Core Operators You Must Know (Beginner Essentials)​

In this chapter, we’ll focus on the three foundational operators:

  1. PythonOperator
  2. BashOperator
  3. EmptyOperator

These alone can build 80% of real-world DAGs.

1️⃣ PythonOperator​

What is PythonOperator?​

PythonOperator executes a Python callable.

Use it when:

  • Processing data
  • Calling APIs
  • Transforming files
  • Writing business logic

Example: Simple Python Task​

 from airflow.operators.python import PythonOperator

def greet_user():
 print("Hello from Airflow!")

Adding PythonOperator to a DAG​

 python_task = PythonOperator(
 task_id="greet_task",
 python_callable=greet_user
 )

Input & Output Example​

Input​

No input parameters

Output​

 Hello from Airflow!

📌 Output appears in task logs inside the Airflow UI.

Passing Data to PythonOperator​

 def greet(name):
 print(f"Hello {name}")

python_task = PythonOperator(
 task_id="personal_greeting",
 python_callable=greet,
 op_args=["Airflow User"]
 )

Output​

 Hello Airflow User

2️⃣ BashOperator​

What is BashOperator?​

BashOperator executes shell commands.

Use it when:

  • Running scripts
  • Calling CLI tools
  • Managing files
  • Triggering jobs

Example: Bash Task​

 from airflow.operators.bash import BashOperator

bash_task = BashOperator(
 task_id="print_date",
 bash_command="date"
 )

Input & Output Example​

Input​

System shell command

Output​

 Mon Dec 16 00:00:00 UTC 2024

Real-World BashOperator Example​

 bash_task = BashOperator(
 task_id="create_directory",
 bash_command="mkdir -p /tmp/airflow_demo"
 )

📌 Great for lightweight system-level tasks.

3️⃣ EmptyOperator​

What is EmptyOperator?​

EmptyOperator does nothing — and that’s its superpower 🦸‍♂️

Use it for:

  • Start/end markers
  • Logical checkpoints
  • DAG readability

Example​

 from airflow.operators.empty import EmptyOperator

start = EmptyOperator(task_id="start")
 end = EmptyOperator(task_id="end")

Putting It All Together (Complete DAG Example)​

 from airflow import DAG
 from airflow.operators.empty import EmptyOperator
 from airflow.operators.python import PythonOperator
 from airflow.operators.bash import BashOperator
 from datetime import datetime

def greet():
 print("Welcome to Airflow Operators")

with DAG(
 dag_id="operators_basics_dag",
 start_date=datetime(2024, 1, 1),
 schedule_interval="@daily",
 catchup=False,
 tags=["operators", "airflow"],
 ) as dag:

 start = EmptyOperator(task_id="start")

 python_task = PythonOperator(
 task_id="python_task",
 python_callable=greet
 )

 bash_task = BashOperator(
 task_id="bash_task",
 bash_command="echo 'Running Bash Task'"
 )

 end = EmptyOperator(task_id="end")

 start >> python_task >> bash_task >> end

DAG Input & Output Flow​

Input​

  • DAG triggered manually or scheduled
  • No external data

Output​

  1. start → SUCCESS
  2. Python log:
 Welcome to Airflow Operators
  1. Bash log:
 Running Bash Task
  1. end → SUCCESS

Best Practices (Professional Grade)​

✅ One operator = one responsibility
✅ Prefer PythonOperator for logic
✅ Keep BashOperator commands simple
✅ Use EmptyOperator for clarity
✅ Name tasks clearly and consistently

Common Beginner Mistakes​

❌ Writing heavy logic inside BashOperator
❌ Forgetting task_id uniqueness
❌ Mixing business logic in DAG files
❌ Overusing Bash when Python is better

Key Takeaways​

  • Operators define work in Airflow
  • PythonOperator runs Python code
  • BashOperator runs shell commands
  • EmptyOperator improves readability
  • Clean operators = maintainable pipelines

Summary​

In this chapter, you learned:

  • What operators are in Airflow
  • How PythonOperator works
  • When to use BashOperator
  • Why EmptyOperator is important
  • How tasks execute inside a DAG

🎯 You now know how Airflow actually gets work done.

What’s Next?​

👉 Task Dependencies
Learn how to control execution order using:

  • set_upstream
  • set_downstream
  • >> and <<