What is the difference between wide and narrow transformations in Spark?

In Spark, transformations are classified as narrow or wide based on whether they require a shuffle. A narrow transformation (e.g., map, filter, flatMap) can be computed on each partition independently; each output partition depends on a single input partition. A wide transformation (e.g., groupByKey, reduceByKey, join, cogroup) requires data to be redistributed across partitions because each output partition may depend on multiple input partitions. The key difference is that narrow transformations can be executed without shuffling, leading to lower network I/O and higher parallelism, whereas wide transformations incur a shuffle, which is expensive in terms of disk and network I/O. For example, map is a narrow transformation because it applies a function to each element locally, while groupByKey is a wide transformation because it needs to bring all values of the same key together on the same partition. Understanding this distinction helps in writing efficient Spark jobs.