What is question 2 in PySpark?

Question 2 in a typical PySpark interview often asks you to perform a join between two DataFrames and handle duplicate column names. In PySpark, you can join DataFrames using the join() method: df1.join(df2, on='key', how='inner'). The 'on' parameter specifies the join key(s), and 'how' can be 'inner', 'left', 'right', or 'outer'. If both DataFrames contain columns with the same name other than the join key, you should rename them before the join or use the suffix parameter in the join to avoid ambiguity. For example: df1 = df1.withColumnRenamed('value', 'value_df1'); df2 = df2.withColumnRenamed('value', 'value_df2'); joined = df1.join(df2, 'id', 'inner'). After the join, you can select the desired columns. The join operation is distributed; its performance depends on the size of the DataFrames and the chosen join strategy. For small DataFrames, a broadcast join (df1.join(broadcast(df2), 'id')) can be used to avoid shuffling. Complexity is roughly O(n log n) for shuffle joins, but broadcast joins are O(n). Properly handling duplicates and choosing the right join type are key to efficient PySpark code.