How would you get the latest order per customer using Apache Spark?

Use Spark’s window functions. Define a WindowSpec that partitions by customer_id and orders by order_date descending. Then compute row_number() over that window. Filter where row_number equals 1 to keep only the most recent order per customer. Example in Scala/Python:

from pyspark.sql import Window
from pyspark.sql.functions import row_number, desc

w = Window.partitionBy('customer_id').orderBy(desc('order_date'))
df.withColumn('rn', row_number().over(w)).filter(col('rn') == 1).drop('rn')

This approach runs in O(n) time and scales across the cluster, avoiding a full sort of the entire dataset. It is the standard pattern for “latest record” logic in big‑data environments.