What is an out-of-memory issue and how do you handle it in Spark?

An out‑of‑memory (OOM) issue in Spark occurs when the JVM heap of an executor or the driver runs out of space while processing data. This can happen during large shuffle operations, caching of big RDDs, or when a single partition contains too much data. To handle it, first examine the stack trace and executor UI to locate the offending stage. Common remedies include: increasing spark.executor.memory or spark.driver.memory; tuning spark.memory.fraction and spark.memory.storageFraction; using broadcast joins for small tables; repartitioning or coalescing to balance data; persisting only necessary data; and enabling dynamic allocation to scale executors up or down. If the data is too large for the JVM heap, consider using off‑heap memory (spark.memory.offHeap.enabled) or moving to a more memory‑efficient data format. Monitoring and iterative tuning are key to resolving OOM errors.