How would you handle late‑arriving data in a streaming pipeline?

Late‑arriving data is common in real‑time streams where events can be delayed due to network latency or processing back‑pressure. To handle it, most streaming engines use a combination of watermarks and allowed lateness. A watermark is a timestamp that indicates the engine has seen all events up to that point; events older than the watermark are considered late. Allowed lateness defines how long the system will wait for late events before closing a window. When an event arrives after the watermark but within the allowed lateness, the engine can update the window state and recompute aggregates. If it arrives after the allowed lateness, it is typically dropped or sent to a side output for later analysis. Additionally, you can use stateful joins with a grace period, or store late data in a separate sink for batch re‑processing. Choosing the right watermark strategy and lateness threshold depends on the business tolerance for stale data and the volume of late events.