Xcom In Airflow May 2026

XCom (short for cross‑communication ) is Airflow’s built‑in mechanism for exchanging small pieces of data between tasks. When used wisely, they unlock powerful patterns. When abused, they break your DAGs. Let’s see how to use them correctly. XComs are key‑value pairs stored in Airflow’s metadata database. A task can push an XCom (write a value under a key), and another task can pull that value (read it).

@task def consume_two(data): return f"Got data['source']" @task def fetch_urls() -> list[str]: return ["http://a.com", "http://b.com"] @task def download(url: str) -> str: # download content return f"content_of_url"

process_record(get_latest_record_id()) @task def produce_data(): return "ids": [1,2,3], "source": "api" @task def consume_one(data): return f"Got data['ids'][0]" xcom in airflow

No xcom_push or xcom_pull needed – the TaskFlow wiring handles it. With traditional operators, you must push/pull manually.

Here, each mapped task gets its own XCom value, and aggregate receives a list of all results. ❌ Passing large data # BAD – will bloat metadata DB @task def bad_task(): return large_dataframe.to_dict() # can be MB/GB ✅ Better: Store data in S3/GCS and pass the path as an XCom. ❌ Pulling from a task that hasn’t run @task def step_one(): return 1 @task def step_two(x): # If step_one failed or was skipped, this will raise an error return x + 1 Let’s see how to use them correctly

Here’s a structured, useful blog post about — written for data engineers who want to move beyond basic tasks and build real DAGs. Mastering XComs in Apache Airflow: Cross‑Task Communication Without the Pain One of the first surprises when learning Airflow is that tasks run isolated from each other. You can’t just set task_2.data = task_1.data . So how do you pass a value from one task to another? XComs .

process(extract()) # XCom passed implicitly catchup=False) def xcom_demo():

from airflow.decorators import dag, task from datetime import datetime @dag(start_date=datetime(2024,1,1), schedule=None, catchup=False) def xcom_demo():