How do you use the explode() function on the 'skills' column in a PySpark DataFrame to create a separate row for each skill while preserving the 'name' column?

In PySpark you can flatten an array column with the explode() function. First import the function: from pyspark.sql.functions import explode. Then apply it in a select or withColumn. For example:

from pyspark.sql.functions import explode

# Assume df has columns name and skills (array of strings)

result = df.select("name", explode("skills").alias("skill"))

This will produce a DataFrame where each element of the skills array becomes its own row, duplicating the name for each skill. If you want to keep the original row structure you can use withColumn:

df = df.withColumn("skill", explode("skills")).drop("skills")

The resulting DataFrame has columns name and skill. Complexity is O(n) where n is the total number of skills across all rows. This is the idiomatic way to explode array columns in Spark.