Home › Interview Questions › Write PySpark code to filter rows where status is …

Write PySpark code to filter rows where status is 'success' and aggregate the total amount per customer using groupBy and sum.

🟢 Easy Coding Fresher level
1Times asked
Oct 2026Last seen
Oct 2026First seen

💡 Model Answer

The provided code demonstrates how to use PySpark’s DataFrame API to filter rows and aggregate data. First, filter(col('status') == 'success') keeps only rows where the status column equals 'success'. Then groupBy('customer_id') groups the remaining rows by customer ID. Finally, agg(sum('amount').alias('total_amount')) calculates the sum of the amount column for each group and names the resulting column total_amount. The resulting DataFrame contains one row per customer with the total amount of successful transactions. The operation is performed in a distributed manner; each partition computes partial sums, which are then combined. The time complexity is O(n) where n is the number of rows, and the space complexity is O(k) for k distinct customers. This pattern is common for summarizing transactional data in Spark.

This answer was generated by AI for study purposes. Use it as a starting point — personalize it with your own experience.

🎤 Get questions like this answered in real-time

Assisting AI listens to your interview, captures questions live, and gives you instant AI-powered answers on a discreet on-screen overlay.

Get Assisting AI — Starts at ₹500