How do you avoid data skew in Spark? What is sorting? What is broadcast join, and can you give an example scenario where you would use it?

Data skew occurs when a few keys dominate a dataset, causing some partitions to process far more records than others and creating bottlenecks. To mitigate skew you can: 1) add a random salt to the key (salting) so that heavy keys are distributed across multiple partitions; 2) use a custom partitioner that balances load; 3) apply a map‑side combine to reduce data before shuffling; 4) use Spark’s built‑in skew join optimization that splits heavy keys into multiple partitions. Sorting is the process of ordering data, often used before a join to enable efficient merge‑joins or to satisfy downstream operations that require sorted input. Broadcast join is a join strategy where the smaller dataset is broadcast to all executors, eliminating the shuffle. A typical scenario is joining a large fact table with a small dimension table, such as joining a 10‑million‑row sales table with a 10‑thousand‑row product lookup. Broadcasting the product table allows each executor to perform the join locally, speeding up the job and reducing shuffle traffic.