Give an example of PySpark code to perform an operation on a DataFrame.
💡 Model Answer
Here’s a simple PySpark example that reads a CSV, filters rows, and shows the result:
from pyspark.sql import SparkSession
spark = SparkSession.builder.appName("Example").getOrCreate()
# Read data
df = spark.read.csv("/path/to/data.csv", header=True, inferSchema=True)
# Perform an operation: filter rows where age > 30
filtered_df = df.filter(df["age"] > 30)
# Show the result
filtered_df.show()This code demonstrates creating a Spark session, loading data into a DataFrame, applying a filter transformation, and displaying the output. You can chain additional transformations, such as groupBy and agg, to compute aggregates: df.groupBy('store_id').agg({'sales_amount': 'sum'}).show() or join with another DataFrame. The example illustrates the core workflow of reading, transforming, and writing data in PySpark.
This answer was generated by AI for study purposes. Use it as a starting point — personalize it with your own experience.
🎤 Get questions like this answered in real-time
Assisting AI listens to your interview, captures questions live, and gives you instant AI-powered answers on a discreet on-screen overlay.
Get Assisting AI — Starts at ₹500